Civilized truths, hateful lies? Incivility and hate speech in false information – evidence from fact-checked statements in the US

Figures & data

Table 1. Results of two rounds of inter-coder reliability tests.

Variable	Source media speaker	Source statement	Source message	Topic	Rating	Modality	Type verification	Media critique	Negativity	Partisan attack	Hate speech	Out-group	Hate attribution	Hate language
Round I
KALPHA	.79	.74	.63	.63	.89	.74	.75	1.000	.65	.81	.78	.75	.64	.75
% Agreement	98	84	82	76	91	89	84	100	80	93	93	89	93	87
Round II
KALPHA	.78	.76	.68	.75	.89	.76	.78	1.000	.72	.78	.76	.76	.68	.76
%Agreement	98	86	84	85	91	89	86	100	85	92	92	90	95	90

Note: All inter-coder reliability tests were conducted on sample frames that are not part of the main study (both are from the same fact-checking platforms). The two rounds were conducted before coding the full sample frame, and there was a delay of about a month between the two rounds. KALPHA – Krippendorff’s alpha.

Table 2. Negativity, incivility and hate speech in different types of (un)truthfulness.

	True	Mostly true	Mostly false	False	Pants on Fire!	Overall
Negative tone	52.2%	43.2%	62.4%	64.0%	57.8%	55.8%
Partisan attacks on Democrats	7.6%	9.0%	18.3%	26.9%	30.4%	17.1%
Partisan attack on Republicans	12.6%	7.5%	10.2%	9.1%	4.9%	9.3%
Media attacks	1.5%	3.0%	3.0%	4.1%	4.9%	3.1%
Hate speech	20.7%	14.6%	24.4%	32.0%	33.3%	24.0%
N	198	199	197	197	102	894

Note: Cell entries represent proportions of all fact-checked claims within the respective category of (un)truthfulness.

Table 3. Logistic regression models predicting likelihood of negative and uncivil language in (un)truthful communication.

	Negativity		Attack on Democrats		Attack on Republicans		Media critique		Hate speech
	B (SE)	OR & 95% CI	B (SE)	OR & 95% CI	B (SE)	OR & 95% CI	B (SE)	OR & 95% CI	B (SE)	OR & 95% CI
(constant)	−.81 (.43)*	.07 [−1.54, .26]	−3.26 (−.11)*	.04 [−4.61, −2.26]	−4.75 (1.97)*	.01 [−6.82, −3.05]	−4.80 (2.98)	.01 [−18.79, −2.92]	−3.53 (.59)*	.03 [−4.67, −2.52]
FC (Snopes)	.29 (.15)	1.34 [−.04, .57]	−.26 (.26)	.77 [−.75, .26]	.07 (.30)	1.07 [−.47, .67]	−.37 (.51)	.69 [−1.43, .74]	.48 (.21)*	1.61 [.08, 1.01]
Topic	.01 (.01)	1.01 [−.02, .03]	.08 (.00)**	1.09 [.04, .13]	.05 (.02)*	1.06[.01, .11]	.16 (.12)	1.07 [.03, .60]	.06 (.01)**	1.06 [.04, .10]
Media source	−.01 (.00)	.99 [−.03, .02]	.01 (.02)	1.01 [−.02, .04]	.04 (.02)	1.04 [.01, .08]	−.00 (.03)	.99 [−.06, 06]	.04 (.01)*	1.04 [.02, .07]
RWP source	−.34 (.29)	.68 [−1.07, .20]	.62 (.32)^†	1.87 [−.07, 1.29]	−.42 (2.09)	.66 [−8.77, 1.45]	.06 (.55)	1.07 [−1.05, 1.10]	−.07 (.35)	.94 [−.68, .71]
Mainstream politicians	.13 (.21)	1.14 [−.35, .47]	−.27 (.28)	.76 [−.82, .31]	1.58 (.30)**	4.87 [.64, 3.02]	−17.96 (.43)**	.01 [−18.62, −16.89]	.16 (.05)	1.18 [−.51, .76]
Journalist source	.11 (−.01)	1.11 [−.50, .85]	.47 (.38)	1.60 [−.28, 1.33]	.55 (3.10)	1.74 [−7.96, 1.80]	−.39 (4.24)	.68 [−18.22, 1.06]	.47 (.35)	1.61 [−.14, 1.24]
Citizen source	−.03 (.35)	.97 [.65, 1.46]	−.02 (.28)	.98 [−.54, .51]	.86 (.32) ^†	2.36 [−.03, 2.31]	−.13 (.52)	.88 [−1.40, .93]	−.22 (.28)	.80 [−.72, .56]
Verification type	.30 (.14)	1.35 [1.04, 1.76]	.01 (.01)	1.01 [−.37, .36]	.37 (.23)	1.45 [−.16, .81]	−.22 (.38)	.80 [−.96, .63]	.11 (.16)	1.12 [−.28, .43]
Mostly true	−.38 (.20)^†	.68 [−.72, 0.01]	.32 (.37)	1.38 [−.53, 1.05]	−.46 (.38)	.63 [−1.31. .11]	.82 (2.99)	2.28 [−.62, 8.94]	−.34 (.31)	.71 [−1.02, .34]
Mostly false	.37 (.21)^†	1.45 [−.07, .82]	.90 (.38)**	2.45 [.27, 1.69]	−.13 (.01)	.88 [−.78, .46]	.47 (2.41)	1.60 [−.86, 8.57]	.23 (.27)	1.26 [−.36, .72]
Completely false	.42 (.21)*	1.51 [.06, .83]	1.30 (.29)**	3.68[.76, 1.99]	−.45 (.32)	.64 [−1.09, .22]	.26 (2.46)	1.30 [−1.09, 8.39]	.72 (.27)**	2.05 [.15, 1.29]
Nagelkerke R^2	.053		.154		.112		.179		.112
χ^2 (11)	35.97***		86.53***		47.44***		39.82***		69.62***

SE = standard error; OR: odds ratio; df = degree of freedom. Two-tailed tests. bootstrapping was performed. Reference category of false information is true information. Unstandardized regression weights. SEs reported between parentheses. ^†p < .010, *p < 0.05; **p < 0.01; and ***p < 0.001. N = 894.