Fuming Mad and Jumping with Joy: Emotional Responses to Uncivil and Post-Truth Communication by Populist and Non-Populist Politicians on Facebook During the COVID-19 Crisis

ABSTRACT

Social networking sites offer politicians an opportunity to mobilize followers through carefully crafted messages appealing to their emotions. We examine the effects of uncivil and post-truth communication of populist and non-populist party leaders on the emotional emoji reactions of social media users during the COVID-19 pandemic crisis. Conveying a disrespectful tone toward the participants and topics of the debate, lying accusations, and incivility have become prominent aspects of contemporary political discourse in many European countries. We combine research on emotional cues in online political communication and the effects of political elites’ messages on social media. We apply manual content analysis (N = 2,549 posts) to study the political communication of Czech political party leaders on Facebook during the COVID-19 pandemic (March 2020 to February 2021), which generated a higher sense of threat and uncertainty in the public. We show that uncivil and post-truth message elements, affiliation with a populist party, and pandemic influenced the volume of emotional interactions with political posts. The article has important implications for the study of how incivility and attacks on truthfulness can influence opinion exchange in public debate or increase societal polarization.

Social networking sites offer politicians an opportunity to interact with citizens and mobilize followers through carefully crafted messages appealing to their emotions (Gerodimos & Justinussen, 2015). Recognizing the potential of emotion-eliciting communication (Brader, 2005), political actors can make use of technological affordances to receive instant feedback and adjust their communication strategies to create messages that have the best potential to generate emotional responses. Besides commenting, sharing, and liking, the inclusion by Facebook of emotional emoji-like icons gave the public the possibility to share immediate emotion-related reactions to political messages (Eberl et al., 2020). The research identified several message characteristics motivating user reactions on social media. However, most of them looked at the engagement reactions (e.g., Bene et al., 2022; Bracciale et al., 2021) and less at specific emotional emoji responses (e.g., Eberl et al., 2020). We know less about specific message features and circumstances that lead users to click on a reaction button.

We examine the effects of uncivil and post-truth communication of populist and non-populist party leaders on the emotional emoji reactions of social media users during the COVID-19 pandemic crisis. Specifically, we investigate whether accusations of untruthfulness and uncivil speech influence users’ decision to press an emotional reaction icon. This is against the background that lying accusations and incivility have become prominent aspects of contemporary political discourse (Kenski et al., 2018; Kluknavská & Eisele, 2023). Those specific messages are features of political discussion and convey a disrespectful tone toward the participants and topics of the debate (Coe et al., 2014; Kenski et al., 2018). Politicians across the political spectrum employ discursive attacks toward opposing politicians and increasingly use uncivil and polarizing rhetoric, dividing society between the truthful us and the lying others in their communication (Hameleers & Minihold, 2022). Constructing truth-relativizing and disrespectful messages may give politicians an effective tool to mobilize supporters through appeals to emotions (Bjarnøe et al., 2020). Beyond message characteristics, we further inquire about contextual circumstances, examining whether the emotional emoji reactions differ depending on the politicians’ affiliation with a populist party and during the COVID-19 health crisis, which generated a higher sense of threat and uncertainty in public (Eisele et al., 2022).

We combine research on emotional cues in online political communication and the effects of political elites’ messages on social media to understand the users’ decisions to press one of the reaction buttons as a way to engage with political posts emotionally. We apply manual content analysis (N = 2,549 posts) to study the political communication of political party leaders on Facebook in the Czech Republic during the COVID-19 pandemic (March 2020 to February 2021). We look at the social media communication of party leaders as they have important roles in the strategic discourse of political parties in public debate (Ceccobelli et al., 2020). The social media realm gives politicians a platform to directly appeal to their followers and people a way to express their opinions and feelings in reaction to political content (Heiss et al., 2019).

The Czech Republic makes a good case for our purposes, given the dynamics of the pandemic and its political management. The country went from “the best in Covid,” as declared by then Prime Minister Andrej Babiš in the summer of 2020, to the country with the most recorded COVID-19 cases per capita in the world by the autumn of 2021 (Buštíková & Baboš, 2020). Though the government introduced strict measures at the beginning of the pandemic, it was eventually criticized for its poor crisis management. The handling of the crisis was later affected by inconsistent communication, upcoming regional elections, and as a consequence, delayed and chaotic measures, as well as a declining public willingness to comply combined with demands for a return to normal life (Navrátil & Kluknavská, 2020). Focusing on a country in Central Europe also brings important empirical evidence, as studies analyzing the effects of communication on social media emotional reactions tend to have a dominantly Western European focus (e.g., Eberl et al., 2020; Zerback & Wirz, 2021).

Emotions and political communication on social media

Emotions are considered an important element of politics and public debate and have a mobilizing potential (Eisele et al., 2022; Wahl-Jorgensen, 2019). Emotions play, for instance, a role in the persuasiveness of political campaigns (Brader, 2005) or populist messages (Wirz, 2018). Negative emotions such as anger or fear mobilize people to vote (Valentino et al., 2011). Positive emotions such as amusement broaden the scope of attention and prompt people to pursue a broader range of thoughts (Fredrickson & Branigan, 2005). Therefore, it is unsurprising that political actors aim to appeal to people’s emotions by carefully drafting political messages to persuade and mobilize their adherents to support or vote for them. Understanding politicians’ communication is especially important in the context of social media, which gives political actors unmediated access to people with similar views and positions (Engesser et al., 2017).

Previous studies focused mainly on online engagement and identified several message characteristics and contextual factors that drive users’ responses on social networking sites. Most of these studies, however, focus on popularity cues in terms of likes, shares, and comments (e.g., Bene et al., 2022; Bracciale et al., 2021; Heiss et al., 2019). For instance, scholars have specifically considered the role of incivility (Rega & Marchetti, 2021) or populist actors (e.g., Blassnig et al., 2019) in users’ online engagement, focusing on the liking and sharing of political posts. Only a handful of studies examined users’ reactions on social networking sites that may relate to people’s emotions (e.g., Zerback & Wirz, 2021).

Reaction emojis as emotional expressions on social networking sites

Social media users respond to news articles or political posts by leaving a comment, sharing the post, or using one of the emoji buttons (Eberl et al., 2020; Jost et al., 2020). The term “emoji” represents a graphic depiction of facial expressions, hand gestures, persons, objects, or activities (Zerback & Wirz, 2021). Facebook extended the like button by adding emotional reaction emojis in 2016, with the aim of bringing users a way to express their immediate low-effort emotion-related responses publicly (Bil-Jaruzelska & Monzer, 2022; Eberl et al., 2020; Eisele et al., 2022). Facebook termed these icons love (represented by a heart emoji), care (heart-hugging compassionate facial expression), haha (laughing facial expression), wow (surprised facial expression), sad (sad facial expression), and angry (angry facial expression).

These mutually exclusive emoji-like icons have become frequently used features in users’ interactions with political posts (Zerback & Wirz, 2021). While not all decisions of social media users to click on an emoji icon necessarily express a genuine emotion, current research suggests that using one of the reaction buttons may be a way to deal with the emotions triggered by online content. In the case of Facebook, using one of the six icons may represent a user’s emotional response or a way to signal to others that one should respond to the posted content emotionally (e.g., Eberl et al., 2020; Jost et al., 2020).

The amount and distribution of users’ emotional emoji reactions vary over time and across political actors and can also be affected by the content of political communication. During the 2016 U.S. presidential elections, love reactions were most common on posts by the U.S. Congress’s Facebook page. However, anger emojis became more prevalent after the elections, particularly in posts attacking political opponents (Hughes, 2018). Eberl et al. (2020) showed that the emotional emoji reactions differed across parties, with populist radical right parties frequently receiving angry reactions, whereas social democrats’ messages prompted love reactions more often. The study by Jacobs et al. (2020) then suggests that populist actors appear to generate more emotional emoji reactions, particularly anger and especially on Facebook. Despite their variance, emotional emoji reactions are frequently used by social media users to respond to political content on social networking sites. Considering the political message characteristics, studies have so far shown how sentiment and perceived issue importance (Eberl et al., 2020), appraisal patterns (Zerback & Wirz, 2021), or populist communication (Jost et al., 2020) affect emotional emoji reactions of social media users.

The effects of political messages on emotional emoji reactions

Though a majority of emotions are triggered by personal and interpersonal contacts, media content and political communication have the ability to elicit people’s emotions, too (Scherer et al., 2010). People experience situations in mediated and political messages indirectly and are usually not affected individually but rather as part of a group (Zerback & Wirz, 2021). Intergroup emotions may arise when people identify with a social group so that they respond emotionally to events or objects that impinge on the group or its identity (Smith & Mackie, 2016).

These intergroup emotions are theoretically based on social identity (Tajfel, 1978) and self-categorization theories (Turner et al., 1987). When people identify with a group, they are more likely to engage in intergroup situations, that is, social comparisons, competition, or conflict between groups, and conform to the norms of the activated group in their beliefs, attitudes, or behaviors, seeing the actions that advance the group’s interests as desirable and beneficial (Smith & Mackie, 2016). When group identification turns into an important social identity, the group takes emotional significance for an individual (Tajfel, 1978). Following appraisal theories (e.g., Frijda, 1986), intergroup situations are appraised in terms of their implication for their social identity. These specific group-based appraisals, evaluations, or interpretations of intergroup situations will determine specific group-based emotions (Smith & Mackie, 2016). Based on these insights, we can assume that the emotional emoji reactions will more likely be experienced by social media users when the political post activates belonging to group membership (those with which people psychologically identify). In the next section, we will overview how incivility and post-truth communication elements, affiliation with a populist party, and crisis situations might affect users’ emotional emoji reactions.

Incivility in political communication

Despite some conceptual discrepancies, scholars have, in general, operationalized uncivil discourses through the lenses of disrespect and impoliteness (e.g., Gervais, 2019). Some studies consider incivility as a continuum that includes profane language as well as more harmful content like hate speech (e.g., Chen, 2017) and threats to democratic norms like negative stereotyping or discrimination (e.g., Papacharissi, 2004). Other scholars suggest distinguishing intolerant and uncivil discourse (e.g., Rossini, 2022) and define incivility as rude, offensive, profane, or heated discourse. In this article, we adopt a definition of incivility as the use of vulgar and insulting language and ridicule which includes invectives, name-calling, ad hominem attacks, mockery, or insults (e.g., Brooks & Geer, 2007; Coe et al., 2014; Kenski et al., 2020; Mutz, 2015; Rossini, 2022; Stryker et al., 2016). These forms of incivility are also considered the most uncivil by audiences (Kenski et al., 2020).

While studies found mixed results in the effects of incivility on attitudes or behavior (e.g., Brooks & Geer, 2007; Van’t Riet & Van Stekelenburg, 2022), the literature suggests that incivility in political discourses may prompt emotions (e.g., Chen, 2017; Phillips & Smith, 2004; Rösner et al., 2016). Other research specifies that when incivility is directed at a person or their in-group, this leads to emotional responses (e.g., Gervais, 2015). By violating social norms (Van Kleef et al., 2015), incivility has been shown to generate negative affective responses (Mutz, 2015) and feelings of anger (Gervais, 2019). However, it can also lead to positive emotions such as enthusiasm (Kosmidis & Theocharis, 2020).

Following intergroup emotions theory, the use of incivility, understood on the basis of a disrespectful and impolite tone, can induce group-level feelings when group identity is salient (Gervais, 2019). When political elites use uncivil messages in relation to a group conflict (i.e., insulting the in-group or out-groups), it likely stimulates group salience, and group-based appraisals lead to emotions. When the people’s own group is targeted with incivility, people may have a need to defend it and thus experience defensive anger or aversion (Gervais, 2019). When the out-group is targeted, the message can activate negative feelings such as fear or anger toward the out-group but might also improve sentiments toward the other side (Gervais, 2015). The use of uncivil messages on like-minded audiences can also theoretically induce a feeling of belonging to a group and thus lead to positive emotional emoji reactions.

H1:

The use of uncivil communication in posts increases the likelihood of emotional emoji reactions to them.

The use of incivility by some actors can theoretically produce more emotional emoji reactions than others. In particular, we consider whether the uncivil language employed by populist actors amplifies the emotional emoji reactions of social media users. Populists put forward a Manichean vision separating society into two antagonistic groups (Mudde, 2007) and tend to include insulting language in their messages (van der Goot et al., 2022). Using uncivil language in relation to a divide between the pure people and corrupt elites can make social identity especially salient and thus elicit emotions. We explore this relationship by posing an open research question:

RQ1:

To what extent does the use of uncivil communication spread by populist actors increase the likelihood of emotional emoji reactions to their posts?

Post-truth communication

Scholars have recently started to point to an increase in the relativization of truth in public communication (e.g., Van Aelst et al., 2017). The shifts in conditions through which people understand lies and determine the truth as a verifiable expression of reality (Farkas & Schou, 2018) have been described as post-truth (Waisbord, 2018), truth contestation (Kluknavská & Eisele, 2023) or communicative untruthfulness (Hameleers & Minihold, 2022). When facts and knowledge become relative and a matter of assertion, objective reality can become a mere construct, and any statement about the world can potentially become truthful (Waisbord, 2018). Such relativization of the truth can manifest in post-truth communication. It is futile to determine whether specific statements accurately represent or distort reality, but it is important to understand how political actors shape and diffuse discourses on truth (Kluknavská & Eisele, 2023).

Post-truth communication is rooted in a binary vision of society and conflict-centered politics, in which the two groups hold their own versions of the truth (Waisbord, 2018). Firstly, it is characterized by anti-elite antagonism expressed through the evaluations of opponents’ truthfulness (Kluknavská & Eisele, 2023). Political actors discursively construct accusations of intentional information manipulation, accusing others of lying and labeling their statements as deliberately false or fake (Hameleers & Minihold, 2022). Secondly, political actors use post-truth discourses to create truer versions of reality, which are framed as real, objective, and unbiased (Kluknavská & Eisele, 2023). Inventing counter-factuality, counter-knowledge, or alternative expertise can become a tactic that turns attention from established knowledge and guides people to believe information that confirms their existing attitudes and beliefs (Hameleers & Minihold, 2022).

Post-truth communication is expressed by the appeal to people’s sentiments and emotions, trying to prevail over reason, arguments, and evidence by shifting the meaning of true and false (Dahlgren, 2018). When political actors accuse others of lying, individuals who identify with a group may feel that they are lying to us and thus feel angry or hurt (Smith & Mackie, 2016). Similarly, when politicians praise an in-group for telling the truth, unveiling the true reality and facts, people may feel good about their own group, and thus positive emotions will be elicited.

H2a:

References to truth and knowledge in posts increase the likelihood of emotional emoji reactions to them.

H2b:

Accusations of untruthfulness in posts increase the likelihood of emotional emoji reactions to them.

Post-truth communication finds a close affinity with populism (Kluknavská & Eisele, 2023). The populist binary interpretation of the world can be interpreted as a divide between the lying elite and honest people (Hameleers et al., 2017). Questioning the truthfulness of other elite actors or the emphasis on own truth and knowledge while expressing in-group threats or attributing blame to out-groups may amplify the identification with a social group and trigger emotions. While the direction of the relationship is unclear, we can theoretically expect that the post-truth messages by populist actors play a role in triggering social media users’ emotional emoji reactions. We thus pose the following questions:

RQ2a:

To what extent does the use of truth and knowledge references by populist actors increase the likelihood of emotional emoji reactions to their posts?

RQ2b:

To what extent does the use of accusations of untruthfulness by populist actors increase the likelihood of emotional emoji reactions to their posts?

The interaction of uncivil and post-truth communication

Expressing incivility in political messages can be linked to post-truth communication (Bennett & Livingston, 2018). Information depicting verifiable reality is likely communicated without disrespecting the debate partner and thus less likely approached with the use of uncivil rhetoric. False information or accusations of lying, on the other hand, have been found to be accompanied by uncivil language (Hameleers et al., 2022). Some scholars even consider lying accusations to be one of the layers of uncivil speech (e.g., Kenski et al., 2018). The attacks on truthfulness in an uncivil way may theoretically produce emotions because the fact they are lying to us is further underscored by using speech that violates social norms. Setting an alternative version of the truth may also be connected to incivility in an attempt to gain more attention and emotionality by emphasizing our truth in a daring, unmannerly way (Hameleers et al., 2022). Yet, we do not know how the dynamics between uncivil and post-truth messages affect users’ emotional emoji reactions. We, therefore, postulate the following research questions:

RQ3a:

To what extent does the use of incivility and truth and knowledge references by politicians increase the likelihood of emotional emoji reactions to their posts?

RQ3b:

To what extent does the use of incivility and accusations of untruthfulness by politicians increase the likelihood of emotional emoji reactions to their posts?

Populist vs non-populist party leaders

The characteristics of a political actor can affect the users’ responses to social media posts. We consider the possible effect of the affiliation of a political leader with a populist party which appears to generate more emotional emoji reactions than a non-populist party (Jacobs et al., 2020). Populism, as a thin-centered ideology, interprets the world through a black-white perspective. Populists separate society into two homogenous, antagonistic groups, the good people and the bad elite, and demand the sovereignty of the people be restored (Mudde, 2007). Making the distance between the deprived in-group and blamed out-groups may increase emotional responses along the identity lines (Hameleers et al., 2017). Appealing to a sense of injustice (Wahl-Jorgensen, 2019) and portraying society as an ultimate struggle of us who are harmed by dangerous them (Mudde, 2007) may invoke a sense of belonging and elicit intergroup emotions of anger, fear, or anxiety aimed at the others or pride, enthusiasm, or love felt toward the own group.

H3:

Posts by populist actors are more likely to receive emotional emoji reactions than those of non-populist actors.

The COVID-19 crisis

Public crises are occurrences of large-scale emergencies with insecure prospects, inducing a sense of threat, uncertainty, and heightened emotions (Eisele et al., 2022). The COVID-19 pandemic was one of modern history’s most disruptive global health challenges to politics, economies, and social life (Wodak, 2021). Politicians played a key role in framing and understanding the situation by the public (Verbalyte et al., 2022). Throughout the pandemic, society’s positive emotions such as hope, gratitude, and pride along with such negative emotions as fear, anger, or anxiety were elevated depending on how measures taken to tackle the crisis, society’s contentment with elites’ responses, and the development of the pandemic itself (Eisele et al., 2022).

Several pandemic waves brought intensive anti-coronavirus measures and introduced strict lockdowns, possibly inducing elevated emotions. Eisele et al. (2022) showed how the first lockdown led to intense emotionality from news users in their comments under pandemic articles. This may include both negative and positive emotions, as people during exceptional events tend to rally behind their leaders and increase support for crisis managers (Bol et al., 2021; Kritzinger et al., 2021). Later into the pandemic, the citizens’ reactions became less supportive (Kittel et al., 2021) and more irritated (Thiele, 2022).

H4a:

Posts during the lockdowns and heightened emergency government measures during the pandemic are more likely to receive emotional emoji reactions from users.

H4b:

The posts that include COVID-19 as a topic are more likely to receive emotional emoji reactions from users.

Methodology

Czechia in the first year of the pandemic

The first three COVID-19 cases were registered in the Czech Republic on March 1, 2020. The initial pandemic wave saw immediate mitigation measures, including strict lockdown, a state of emergency adopted on March 12, 2020, and unprecedented public solidarity, which helped to slow down the pandemic (Buštíková & Baboš, 2020; Navrátil & Kluknavská, 2020). Resulting low numbers of confirmed cases and growing public demand for a return to normal life led to the cancellation of the restrictive measures by early summer 2020, after which then-Prime Minister Andrej Babiš declared that the country was “best in covid” (Buštíková & Baboš, 2020).

The subsequent COVID-19 waves were marked by public criticism over the measures, delays in vaccination, and questions over competency to handle the pandemic. The Czech regional and Senate elections in October 2020 delayed new measures to tackle the second pandemic wave, after which the country recorded more new cases per million inhabitants than any other country in the world (Bartoníček et al., 2022). By the beginning of March 2021, the country reached the highest per capita infection and death toll rate in the world, putting the healthcare system under immense pressure. Only after facing harsh public criticism over lax measures and chaotic communication (e.g., Červenka, 2021), the government introduced the strictest lockdown to that date in March 2021.

Overall, the first year of the pandemic in Czechia, while starting at a high level, was characterized by a downward trend in public support for the government and its crisis mitigation measures. Mirroring increased public crisis fatigue and frustration, our period of analysis thus promises wealthy material for a study interested in public emotions and their expression on social media in response to politicians’ direct communication.

Sample and coding

The data for our study consists of a corpus of social media posts from the Czech parliamentary party leaders’ Facebook fan pages. Generally speaking, Facebook is the most popular social media network in the Czech Republic with 70% of respondents in a 2021 survey indicating that they used Facebook. In terms of news use, Facebook is by far the most popular social media network (42%), followed by YouTube (21%) (see Štětka, 2021). It is thus an important communication channel for politicians speaking to the Czech public. Our sample includes nine party leaders, of which two were leaders of governing parties and seven were leaders of opposition parties (see Table 1). We utilized the CrowdTangle data collection tool (CrowdTangle Team, 2020) to download all Facebook posts (N = 10,207) by party leaders during the first year of the COVID-19 pandemic from March 2020 to February 2021. The data downloaded include the messages that politicians shared on their pages and the reactions (e.g., love, sad, angry) of people to these posts. To keep the coding manageable, we coded a stratified random sample (25%) for each politician (n = 2,549) to cover the whole period.

Table 1. Czech party leaders included in the analysis.

Leader	Party Affiliation	Populist	Government	Number of Posts Included	Total Number of Posts
Andrej Babiš	Action of Dissatisfied Citizens (ANO)	Yes	Yes	427	1,710
Tomio Okamura	Freedom and Direct Democracy (SPD)	Yes	No	601	2,405
Jan Hamáček	Czech Social Democratic Party (ČSSD)	No	Yes	75	300
Vojtěch Filip	Communist Party of Bohemia and Moravia (KSČM)	No	No	95	381
Petr Fiala	Civic Democratic Party (ODS)	No	No	303	1,212
Vít Rakušan	Mayors and Independents (STAN)	No	No	150	602
Marian Jurečka	Christian and Democratic Union – Czechoslovak People’s Party (KDU-ČSL)	No	No	413	1,654
Markéta Pekarová Adamová	TOP 09	No	No	207	831
Ivan Bartoš	Czech Pirate Party (Piráti)	No	No	278	1,112

Note. As Ivan Bartoš does not have a Facebook page, we substituted it by drawing on the Czech Pirate Party’s Facebook page.

To identify the political message elements, we conducted a quantitative content analysis of the leaders’ Facebook posts. Two intensively trained independent coders coded the material using a detailed coding scheme. Following similar studies showing the importance of the textual content of the messages in eliciting emotions (e.g., Jost et al., 2020; Widmann, 2022), we coded the textual part of the political posts. We conducted several rounds of intensive training and an inter-coder reliability test, which yielded satisfactory results (Krippendorff’s α: incivility = 0.76, evaluation of truthfulness = 1.00, truth and knowledge references = 0.84, COVID-19 issue = 0.85). Reliability results are also available in Table A in the online supplementary materials.

Dependent variable: Emotional emoji reactions

Following similar research (e.g., Eberl et al., 2020; Jacobs et al., 2020; Jost et al., 2020), our dependent variable is emotional emoji reactions. We operationalized it as a sum (an absolute number) of emotional emoji-like icons, which include Love, Care, Wow, Haha, Sad, and Angry reaction buttons. The emotional emoji reaction count ranged from 0 to 37,887 (M = 937.7, SD = 1986.1). These emoji reactions capture the overall emotionality of the users as they include both positively and negatively valenced reactions. We have decided to look at the overall emotionality as the crisis context provided an environment that filled the public space with emotions ranging from fear, anger, anxiety, or sadness associated with the outbreak of an unknown disease and feelings of antagonism and dissatisfaction with how the crisis is handled, to calls of support, hope, and trust in national leaders’ responses to the pandemic (Vemprala et al., 2021). Appendix B in the online supplementary materials includes a full set of results for each emoji reaction. We do not include like into our dependent variable (see those results in Table B1 of the online supplementary materials), as research has treated it as a popularity cue representing user engagement and expressing, for instance, an agreement with the post’s content (Blassnig et al., 2021).

Independent variables: Incivility and post-truth in politicians’ communication

To assess our hypotheses about incivility (H1), we created a dummy variable for the presence of uncivil language in the post. Incivility is recognized based on the presence of two specific signs: vulgar, pejorative, or derogatory language and insults (against people, institutions, organizations, and policies but also without any object), including name-calling, derogatory, demeaning language aimed at personal characteristics or behavior. Examples of incivility include “a jerk,” “stupid commercial,” “do-gooders,” “parasites,” or “crazy attacks.” Post-truth communication refers to 1) truth evaluation (H2b): an accusation of untruthfulness, lying, untruth-telling, deceiving, misleading the people, society, or others, creating false content (e.g., the media), sharing or creating propaganda, being fake news, or being ideologically biased (and bringing false information as a result of this), and 2) reference to truth or knowledge in the post (H2a): any reference to truth, reality, reason, facts, knowledge, expertise, information, or evidence. Examples of truth evaluation include “a lie spread by pro-Brussels journalists,” “they are making up numbers of infected people,” “the mainstream media lie, lie, and lie,” or “government propaganda.” References to truth or knowledge include examples such as “reality is totally different,” “whether this is the truth,” “these are pure facts,” or “experts claim.”

Independent variable: Populist vs non-populist leaders

To assess the party type (H3) to which a political leader belongs, we classified leaders according to their affiliation with populist or non-populist parties based on the PopuList project’s framework (Rooduijn et al., 2019). Information on the party leaders, their party affiliation, their classification as populists, membership in the government, and the number of their posts included in the analysis is provided in Table 1.

Independent variable: The pandemic context

To account for the context of the COVID-19 pandemic, we identified lockdown (H4a) periods (Government of the Czech Republic, 2020) and included them as a dummy variable. In addition, we also included in the manual content analysis (see earlier description on coding) if the post dealt with COVID-19 as a topic or not (H4b).

Control variables

Following similar research (e.g., Eberl et al., 2020), we included the log-transformed length of the text in the post as well as the type of post, i.e., if the post contained a link, photo, video, or only text. The post can be comprised of only a text, known as a status, or can include a link, photo, or video. Visual posts can, but do not need to, include any text. While we did not code any visual content in those posts, we can control for the potential influence of the presence of visuals in the message. Posts that did not contain any words at all (n = 45) were dropped from the analysis to avoid having contingent units, such as posts containing only a video or photo. Such content could distort our analysis since we only understand the influence of the specific photo or video modality and not the influence of uncivil or post-truth content.

In addition, we also added the log-transformed number of followers at the time when the post was published to control for the popularity of the Facebook page. The Senate and Regional Elections did influence the behavior of the government and electoral campaigning, and the elections are generally found to be an intense time in terms of party competition (Baumann et al., 2021). Therefore, we also included a dummy variable, including the period of 4 weeks in the run-up to the second round of elections on October 9–10, 2020 and the affiliation with the government.

Analysis

For testing the formulated hypotheses, we relied on negative binomial regression analysis as the dependent variable contains overdispersed count data, thus with the variance being larger than the mean. For all models, we calculated variance inflation factors to control for multicollinearity; for none of the models, variance inflation factors were higher than 3.5, thus not indicating issues in this respect. Descriptive statistics for all included variables are shown in Table 2.

Table 2. Descriptive statistics of variables included in the analysis.

Variable Name	Description	Min	Max	Mean	SD
Emotions	Absolute number of reactions using an emoji reaction button (excluding like)	0	37,887	937.704	1986.072
Incivility	Use of uncivil language (yes/no)	0	1	0.108	0.310
Truth/knowledge reference	Reference to truth or knowledge (yes/no)	0	1	0.291	0.454
Truth evaluation	Presence of evaluation of truthfulness in the post (yes/no)	0	1	0.049	0.216
Populist	Post created by the populist actor (yes/no)	0	1	0.403	0.491
Government	Post created by the government actor (yes/no)	0	1	0.197	0.398
Lockdown	Post created during the COVID-19 lockdown (yes/no)	0	1	0.435	0.496
COVID-19 issue	COVID-19 mentioned in the post (yes/no)	0	1	0.142	0.350
Link	Political post includes a link	0	1	0.123	0.329
Photo	Political post includes a photo	0	1	0.570	0.495
Video	Political post includes a video	0	1	0.267	0.443
Status	Political post consists of a text only	0	1	0.040	0.196
Length in words	Count of words in the post (included as log-transformed)	0	825	113.110	162.456
Followers	Count of followers on a posting day (included as log-transformed)	2201	280484	137760.4	108065.2
Elections	Post created during Senate/Regional election campaign period (September 12 to October 10, 2020)	0	1	0.109	0.312

Results

For our baseline model, we tested hypotheses not including interaction terms. As shown in Table 3, all independent variables show significant positive effects, thus generally confirming our hypotheses. The presence of incivility and references to truth and knowledge in the political posts, as well as evaluations of the truthfulness of other political actors in the political posts on Facebook, significantly increased the likelihood of emotional emoji reactions by social media users. The same is true for the posts created by populist actors, the political posts published during the COVID-19 lockdowns, and the political posts which discussed COVID-19. Facebook posts also generated significantly more emotional emoji reactions when they included a link, photo, or video when compared with posts consisting of only a text (reference category). The volume of emotional emoji interaction was also significantly influenced by the page’s number of followers and the post’s length. However, the size of the effect for the latter is very small. Election periods did not generate more emotional emoji reactions.

Table 3. Result of negative binomial regression for emotional emoji reactions.

	Dependent variable: Emotional Emoji Reactions
	Baseline	Incivility	Populism
	(1)	(2)	(3)
Incivility	1.704***	1.730***	2.558***
	(0.078)	(0.112)	(0.118)
Truth/knowledge reference	1.169**	1.151*	1.156*
	(0.058)	(0.062)	(0.068)
Truth evaluation	1.573***	1.843***	1.999***
	(0.106)	(0.124)	(0.140)
Populist	1.568***	1.565***	1.750***
	(0.082)	(0.082)	(0.091)
Government	1.433***	1.441***	1.413***
	(0.079)	(0.079)	(0.080)
Lockdown	1.289***	1.296***	1.290***
	(0.046)	(0.046)	(0.046)
COVID-19 issue	1.433***	1.400***	1.372***
	(0.068)	(0.068)	(0.067)
Link	1.514***	1.487***	1.521***
	(0.074)	(0.074)	(0.074)
Photo	2.052***	2.037***	2.033***
	(0.130)	(0.130)	(0.129)
Video	2.104***	2.058***	2.103***
	(0.080)	(0.080)	(0.079)
Length	1.063**	1.063**	1.093***
	(0.022)	(0.022)	(0.022)
Followers	1.628***	1.630***	1.662***
	(0.032)	(0.032)	(0.032)
Elections	0.981	0.981	0.992
	(0.074)	(0.073)	(0.073)
Incivility x Truth/knowledge reference		1.101
		(0.150)
Incivility x Truth evaluation		0.477**
		(0.239)
Populist x Incivility			0.468***
			(0.154)
Populist x Truth/knowledge reference			0.915
			(0.109)
Populist x Truth evaluation			0.541**
			(0.212)
Constant	0.754	0.752	0.522
	(0.338)	(0.338)	(0.339)
Observations	2,504	2,504	2,504
Log Likelihood	−18,693.560	−18,689.190	−18,674.740
Akaike Inf. Crit.	37,415.120	37,410.390	37,383.480

Note. *p < .05; **p < .01; ***p < .001. Odds ratios (standard error in parenthesis). The reference category for government is opposition, and the type of post is “status.”

The addition of two interaction terms in the second model does not significantly alter results regarding the variables included in the baseline model. Only the interaction of incivility and the evaluation of truthfulness (e.g., an accusation of untruthfulness) of other political actors in Facebook posts shows a significant and negative effect. Zooming in on these interaction terms by plotting marginal effects allows a more fine-grained interpretation. As shown in Figure 1b, truth evaluations of other actors yield more emotional emoji reactions when the posts do not include the use of uncivil language; the presence of incivility also yields more emotional emoji reactions when there are no truth evaluations included in the post.

Figure 1. Marginal effects plots for interaction terms model 2. (a) Incivility x truth and knowledge reference. (b) Incivility x truth evaluation.

For our third model, three interaction terms were included to gain a more nuanced understanding of the dynamics of populism with incivility as well as post-truth elements (references to truth and knowledge and truth evaluation). The interaction of populism and incivility, as well as the interaction of populism and truth evaluation, are significant; also here, we plotted their marginal effects to understand the dynamics better in Figure 2.

Figure 2. Marginal effects plots for interaction terms model 3. (a) Populist x incivility. (b) Populist x truth and knowledge reference. (c) Populist x truth evaluation.

Regarding the interaction of populism and incivility in Figure 2a, non-populist politicians being uncivil get much more emotional emoji reactions than when they do not include uncivil language in their posts. It does not make a significant difference if populists include or do not include uncivil language in their posts. No significant difference is shown for posts including uncivil language. Populists get more emotional emoji reactions than non-populists when not using uncivil language. While the interaction of a reference to truth or knowledge and populism is not significant, it shows that populist leaders get more emotional emoji reactions than non-populist leaders, as shown in the base model in Figure 2b. As shown in Figure 2c, non-populists, including truth evaluations in their posts, get much more emotional emoji reactions; populists get more emotional emoji reactions than non-populists when not evaluating the truth in their posts.

Discussion and conclusion

This study analyzed the effects of Czech party leaders’ political communication on emotional emoji reactions during the COVID-19 pandemic crisis. With growing concerns over political elites’ use of unnecessarily disrespectful rhetoric toward the discussion forum (Coe et al., 2014), we first investigated two possible antecedents of emotional emoji reactions of users on Facebook: uncivil and post-truth message characteristics. While some users may use emotional emoji reactions independently of the post’s content, we found that employing incivility or post-truth characteristics in social media messages pays off in terms of eliciting people’s emotional emoji responses.

These message features are individually likely to receive more emotional emoji reactions from users, but attacking the truthfulness of the opponent seems to elicit more reactions when uncivil language is not present in their social media posts. Moreover, applying uncivil and post-truth communication seems to be a good strategy, particularly for non-populist politicians. Inquiring about the influence of contextual circumstances, our findings suggest that populist leaders received more emotional emoji reactions from their social media followers than non-populist leaders (Jacobs et al., 2020) and the COVID-19 played a role in elevating emotional emoji reactions to pandemic-related political messages.

These findings uncover that emotional popularity cues can, to some extent, be explained by content characteristics. These findings highlight the importance of such cues for populist and non-populist political actors who want to adjust their party communication to generate the desired responses from their adherents, especially during crises full of fear and uncertainties, when people rely on political leaders’ statements (Verbalyte et al., 2022). However, knowingly including disrespectful and harsh features to strategically increase the visibility and thus perceived relevance of a post among users (Kosmidis & Theocharis, 2020; Mutz & Reeves, 2005) can have mixed consequences for political and public discussion.

The fact that non-populist politicians elicit more emotions by using ill-mannered rhetoric is understandable, as it arouses greater interest, amusement, and outrage among their followers (Sydnor, 2018). Still, it can also lead to the spillover effect of incivility in social media debates (Eberl et al., 2020). Post-truth messages can also further foster societal polarization between those who feel right and those who are wrong (Hameleers & Minihold, 2022; Kluknavská & Eisele, 2023) and make it harder to differentiate between actual and fabricated reality constructed by authority figures. The finding that attacking truthfulness elicits more emotional emoji reactions from users when presented in a civil manner should be taken as a warning sign. Questioning truthfulness while adhering to norms of respectful behavior can more effectively challenge the opponent’s claims and convince others of one’s legitimacy. However, the use of uncivil language may mobilize and engage people in opinion exchange by arousing emotional response, which is beneficial for a healthy democracy (e.g., Coe et al., 2014; Herbst, 2010; Kosmidis & Theocharis, 2020).

Our findings also indicate that the periods of national lockdowns and posts bringing up the COVID-19 issue increased the emotional emoji reactions of social media users. As suggested by other studies (e.g., Eisele et al., 2022), people’s emotions in response to unprecedented and uncertain situations are important aspects to consider in managing the crisis and getting people to comply with government measures. The results, though not hypothesized, reveal that government actors received increased emotional emoji reactions from their followers. This suggests that the way executives frame the crisis plays an essential role in how people perceive the situation, possibly increasing societal cleavages and polarization at large (Eisele et al., 2022; Widmann, 2022).

There are several limitations to our study. While focusing on overall emotional emoji reactions to political messages makes theoretical sense, especially during a crisis inducing a pool of potentially overlapping positively and negatively valenced responses, future studies could look at a more nuanced understanding of specific reactions. This would help to disentangle how different communication aspects affect distinct emotions. Our study is also restricted regarding its sample size based on manual content analysis. Given the rapid development in automated content analysis, future studies could use automated approaches to assess the content of political communication, possibly looking into the different (social) media platforms over a longer period beyond the crisis context. Considering swift advances in visual political communication, future studies could consider the possible effect of visual material on social media users’ emotional reactions through emojis.

Additionally, platform algorithms, specifically non-neutral algorithmic processes, can influence how users see and interact with political content (Klinger & Svensson, 2018). Though beyond our study to control for this issue, future studies could focus on how algorithms affect the visibility of uncivil and post-truth communication depending on the behavior of users on social networking sites, which may, in turn, affect the emotional responses to these posts. Future research could also look into the mediating role of emotions in using truth accusations and alternative truths in delegitimizing political opponents.

Acknowledgments

We would like to thank Petro Tolochko for his help and feedback.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The work on this article by Alena Kluknavská and Martina Novotná was supported by the NPO ”Systemic Risk Institute” no. LX22NPO5101, funded by European Union – Next Generation EU (Ministry of Education, Youth and Sports, NPO: EXCELES).

Notes on contributors

Alena Kluknavská

Alena Kluknavská is an assistant professor at the Department of Media Studies and Journalism at Faculty of Social Studies at Masaryk University in Brno. She focuses on populist communication, post-truth politics, the populist radical right parties in Central and Eastern Europe, and the communication of social movements. She obtained her PhD in August 2015 from Comenius University in Bratislava.

Martina Novotná

Martina Novotná is a PhD candidate at the Department of Media Studies and Journalism at Faculty of Social Studies at Masaryk University. Her research focuses on informal cross-cutting political talk online, emphasizing incivility, and intolerance.

Olga Eisele

Olga Eisele is an assistant professor at the University of Amsterdam. Her research focuses on political communication with a focus on political crisis communication, the European Union, as well as automated content analysis. She obtained her PhD from the University of Vienna in 2017.