Differential Racism in the News: Using Semi-Supervised Machine Learning to Distinguish Explicit and Implicit Stigmatization of Ethnic and Religious Groups in Journalistic Discourse

ABSTRACT

News coverage plays a crucial role in the formation of attitudes toward ethnic and religious minority groups. On the attitudinal level, it is an established notion that individuals’ explicit and implicit judgments of the same groups can vary. Yet, less is known about the prevalence of implicit group judgments in news coverage. Focusing on a large variety of ethnic and religious minority groups in Germany, the present study sets out to fill this gap. We use semi-supervised machine learning to distinguish explicit and implicit stigmatization of ethnic and religious groups in German journalistic coverage (n = 697,913 articles). Findings suggest that groups that are associated with less wealthy countries, and with culturally more distant countries, face more stigmatization, both explicitly and implicitly. Yet, the data also show that groups associated with Islam and groups with large refugee populations living in the country of study are implicitly, but not explicitly stigmatized in news coverage. We discuss these and other resulting patterns against the backdrop of sociological and psychological intergroup theories and reflect upon their implications for journalism.

KEYWORDS:

• Race
• ethnicity
• stigmatization
• journalism
• news
• automated content analysis

Social psychology has long established the distinction between implicit and explicit stereotypes (Devine, 1989; Dovidio et al., 2002): While individuals are often hesitant to express prejudices explicitly due to social norms against overt discrimination, their behaviors are often shaped by implicitly held group judgments (Devine, 1989). This distinction is therefore useful to explain how discriminatory behaviors and implicit aversions can persist despite anti-discriminatory social norms (Vuletich & Payne, 2019). This notion might also help explain the occurrence of group-related stigmatization in media coverage: If journalists (just like other individuals) held implicit negative attitudes toward ethnic and religious groups, these stereotypes might consciously or subconsciously enter their reporting. Consequently, news discourse would include implicit group-related stigmatization which could foster the societal establishment of group-based racism – even if spreading stigma was not the original intention of journalists who produced respective messages.

Methodologically, the systematic detection of implicit patterns within texts poses a considerable challenge. While there is a plethora of quantitative research investigating more or less explicit patterns of ethnic or racial discrimination within the news (for an overview, see Eberl et al., 2018; Schemer & Müller, 2017; Ter Wal, 2002), implicit stigmatization within news can easier be detected using qualitative rather than quantitative research designs (but, see, Kroon et al., 2021). Focusing on a large variety of ethnic and religious minority groups in Germany and their portrayals in a broad sample of German news coverage, the present study offers large-scale data on the phenomenon. We make use of semi-supervised machine learning by applying two different methodologies that allow distinguishing explicit and implicit stigmatization of groups: latent semantic scaling (Watanabe, 2021) and word embedding biases (Caliskan et al., 2017). In doing so, we do not only explore group-specific amounts of explicit and implicit stigmatization of different groups within media coverage. We also try to detect group-level factors (such as cultural distance between groups, outgroup economic status, and number of outgroup members living in a country) that predict different levels of explicit or implicit stigmatization in the news, linking political communication research with sociological and social-psychological insights on the origins of structural racism.

Race/Ethnicity Within the News as a Function of Cultural Context

When researching racial and ethnic stigmatization within the news, cultural context matters at great lengths. Research focusing on European news coverage has to deal with the paradoxical observation that race, within European countries, is an “absent presence” (Balkenhol & Schramm, 2019; Schemer & Müller, 2017): Mentioning the term “race” is itself a taboo in many countries after the crimes committed by the Nazi regime within the early 20^th century. Even far-right movements avoid invoking race, as any connection to racial essentialism is broadly seen as delegitimizing (Gingrich, 2004). At the same time, racially motivated crimes caused by a “culture of hostility” toward minority populations remain a persistent problem (Piatkowska & Hövermann, 2019), and the rise of populist far-right and anti-immigrant movements (Wodak, 2015) feed the perception of “concepts of race vanishing, movements of racism rising” (Gingrich, 2004). This pattern has been described as a shift toward ethnicism (van Dijk, 1991, p. 25): With racial markers of difference delegitimized, dominant groups shift toward cultural, religious, and ethnic identities to identify othered groups.

This has important implications for researching racism and ethnicism within European news: Unlike the US, where race is regularly mentioned within reporting, and differential treatment of groups can be measured along these lines (Dixon & Linz, 2000), within most European news only people’s ethnicity is typically mentioned. Racial conflict, at least within the mainstream press, is debated in terms of migration, ethnicity, and culture (Eberl et al., 2018), and government data does not include “race” as a category, as is the case within most countries (Balestra & Fleischer, 2018). But, while “race” as a category is absent, that does not mean racism as group-based domination along racial lines is as well (van Dijk, 1991, Goldberg, 2006). In order to measure how news reproduces these hierarchies, we need to measure how different ethnic groups are framed within reporting – and how these markers of identity are linked back to the maintenance of cultural and economic dominance.

Explicit and Implicit Stigmatization of Ethnic and Religious Groups in the News

Research from social psychology found discrepancies between explicit and implicit attitudes toward ethnic groups (Carter & Murphy, 2015). Often, individuals reject the expression of explicit prejudice toward groups, whilst still having implicit aversions. Explicit racism is mostly equated with the open expression of derogatory terms, hate speech, stereotypes, and prejudice, while implicit attitudes are automatic, mostly subconscious, negative associations with social groups (Gawronski & Bodenhausen, 2006). While most people are able to communicate their explicit attitudes and understand the influence these attitudes have on their behavior, they are typically unaware of their implicit attitudes (Dovidio et al., 2002). Even if implicit bias does not imply intentional hostility, implicit negative attitudes can nonetheless evoke discriminatory behavior toward group members (e.g., Devine, 1989).

Prior psychological research has argued that, like other individuals, professional journalists are not free from implicitly stigmatizing outgroup judgments (which they might be aware or unaware of) and that therefore implicit judgments might be (consciously or unconsciously) reflected in their coverage (Marotta et al., 2019). Previous studies have extensively explored stigmatization of ethnic groups within media messages (for an overview, see Eberl et al., 2018; Schemer & Müller, 2017; Ter Wal, 2002): They suggest that minority groups are often under-represented or marginalized in other contexts. If these groups are referred to, they are often portrayed as passive agents or victims. Beyond that, many previous studies have gathered evidence on the framing of ethnic and religious groups as an economic, cultural, or security threat (e.g., C. T. Harris & Gruenewald, 2020; Dixon & Linz, 2000; Kakavand & Trilling, 2022; Meeusen & Jacobs, 2017; Strömbäck et al., 2017). Other research investigated the valence that an ethnic or religious group is associated with in news coverage (e.g. van Klingeren et al., 2015). By and large, negative portrayals of ethnic and religious outgroups appear to prevail in political news content. Yet, extant quantitative research is mostly limited to explicit negative evaluations.

More fine-grained tools to assess implicit biases at a larger scale have become available in recent years. Word embedding has been applied to studying ethnic stereotypes within news coverage (Kroon et al., 2021). However, it can be argued that word-embedding models necessarily contain biases that resemble implicit attitudes present in the authors of the texts that the models are trained on (Caliskan et al., 2017). Automated measurement has been suggested in order to reduce biases, even though fully unbiased solutions are only a theoretical option (Caliskan et al., 2017). By comparing biased and bias-reduced word embedding models, implicit positive and negative connotations within media texts can be assessed (Chan et al., 2021). This allows the comparison of explicit and implicit evaluations of entities such as ethnic and religious groups.

Stigmatization in the News as a Function of Group-Related Factors

In line with previous research from social psychology, it can be expected that implicit and explicit stigmatization of ethnic minority groups within the news will vary as a function of different group-level factors. In the following, we will discuss a number of important factors in more detail and will elaborate on how each of them might play into the occurrence of explicit and implicit group stigmatization in political news.

Economic Group Status

Research into opposition to immigration shows that there is a clear connection between anti-immigration sentiment and specific immigrant groups’ economic status (Ben-Nun Bloom et al., 2015; Ruedin, 2020). At the same time, the underlying mechanism is less clear: Initial studies suggested individual resource competition with migrants as the driving force behind anti-immigrant sentiment (Malhotra et al., 2013), while recent findings emphasize that concerns about detrimental economic effects at the society-level motivate opposition to migration (Hainmueller & Hopkins, 2014). A third strand, meanwhile, emphasizes that outgroup stigmatization can be seen as a mechanism for the maintenance of global economic inequalities, even if they do not serve an individual’s economic self-interest (Kustov, 2019; Sidanius et al., 2004). As Połońska-Kimunguyi (2022) argues, economic threat narratives can also be part of the racialization of migrant groups by describing them as innately inferior and undeserving of economic status.

Kustov (2019) demonstrates that GDP per capita of outgroups’ countries of origin lowers perceived group threat. This is consistent with evidence indicating that the status of countries of origin explains anti-immigrant attitudes (Gheorghiu et al., 2021). Following this rationale, it can be expected that ethnic groups which are associated with origin countries of lower economic status will be more stigmatized in news coverage as well.

H1:

GDP per capita of an origin country will be positively related to the explicit sentiment in news portrayals of ethnic groups that can be associated with that country.

At the same time, group hierarchy theory suggests deep-seated beliefs motivate prejudice against low-status outgroups, which could also inform implicit sentiments. However, since there is less empirical evidence for this assumption, we open-endedly ask:

RQ1:

How is GDP per capita of an origin country related to the implicit sentiment in news portrayals of ethnic groups that are typically associated with that country?

Outgroup Size

Social-psychological research yielded contradictory results regarding the effects of group size within a host country on sentiment toward outgroups. Intergroup contact theory suggests that hostility toward outgroups is lowered if more people encounter group members in their everyday life (Pettigrew, 1998). Indeed, everyday contact with outgroups is correlated with reduced threat perceptions (Green et al., 2020). Yet, perceived group size can also increase perceived group threat (Gorodzeisky & Semyonov, 2020). It has even been suggested that actual intergroup contact might lower perceptions of group size which in turn lowers group threat (Heath et al., 2020). With these contradictory effects in play, we ask:

RQ2:

How is the number of group members living in a country related to (a) the explicit and (b) the implicit sentiment of ethnic groups´ news portrayals?

Reasons for Migration

If intergroup contact and economic threat perceptions can shape the portrayal of an outgroup within the news, another important factor should be the share of involuntary immigration: Refugee migrants enter a host country usually without economic capital and with immediate humanitarian needs, and consequently face stronger difficulties with regard to integration. According to integrated threat theory (Stephan et al., 2008) this could heighten, for instance, economic or security threat perceptions. Moreover, previous content analyses found that refugee groups are being portrayed particularly negative in media coverage, for example as criminals (e.g., Chouliaraki & Zaborowski, 2017; Gardikiotis, 2003). Yet, since it is unclear from the current state of research in how far this might affect implicit and explicit stigmatization differently, we ask:

RQ3:

How is the number of group members living in a country for refugee, asylum, or international protection reasons related to (a) the explicit and (b) the implicit sentiment of ethnic groups´ news portrayals?

Cultural Proximity

Besides economic threat and intergroup contact, cultural threat perceptions also influence prejudice toward outgroups (Tartakovsky & Walsh, 2020). Consequently, support of exclusionary policies focused on specific ethnic groups is informed by both cultural and economic threat perceptions, with Muslims and Roma facing the largest opposition among Europeans, while inter-European migration provokes comparatively less opposition (Gorodzeisky & Semyonov, 2019). We expect threat perceptions to be informed by ingrained cultural differences (Kakavand & Trilling, 2022). Perceived cultural differences can also feed narratives of supposed cultural inferiority (Połońska-Kimunguyi, 2022), and the shift from openly racist concerns over racial homogeneity to concerns over cultural homogeneity can be part of a shift from racist to ethnicist hierarchization within public discourse (van Dijk, 1991, p. 102).

Thus, differences in collectively held values between different societies should be a good indicator of these countries’ cultural distance. As these measures have already been successfully applied in content analytical research (Sheafer et al., 2014), we will use them as the basis for cultural distance. We expect:

H2:

News portrayals of ethnic groups that are typically associated with Western culture will have more positive (a) explicit and (b) implicit sentiment than ethnic groups that are typically associated with non-Western cultures.

Religious Identity

Opposition toward immigration in European countries and the U.S. is typically highest toward Muslim-identified groups (Gorodzeisky & Semyonov, 2019). Ethnic groups associated with Islam are regularly used as the exemplar case of culturally distant groups within the research literature (Bleich et al., 2022; Kustov, 2019). But, research indicates that Islamophobia is not just driven by cultural distance, but additionally by Muslims’ status as a suspect community which causes them to be falsely associated with Islamic terrorism (Obaidi et al., 2018). Additionally, media portrayals of Muslims within Germany reproduce racial stereotypes of criminality and sexualized threat (Yurdakul & Korteweg, 2021) with narratives that can be historically traced back to colonial narratives of Muslim invasion (Gingrich, 2004). Therefore, to distinguish cultural distance from other religion-related effects, we argue:

H3:

News portrayals of ethnic groups that are typically associated with Islam will have more negative (a) explicit and (b) implicit sentiment than ethnic groups that are not typically associated with Islam.

Method

To explore our pre-registered hypotheses and research questions¹, we conducted an automated content analysis of news articles from a broad range of established news outlets from Germany (n = 697,913 articles). We used two different approaches of semi-supervised machine learning to detect implicit and explicit stigmatization of a variety of ethnic and religious minority groups based on the co-occurrence of group labels with positive (admiration) and negative (fear) emotional judgments in news coverage.

Sample

This study is based on a secondary analysis of a sample scraped online from 2017-04-10 to 2018-04-10 (Freudenthaler & Wessler, 2022). As we study quite universal patterns of explicit and implicit stigmatization of groups within texts that should change over decades rather than years, we deemed it justifiable to rely on this analysis-ready corpus which, however, does not cover most recent years. In total, the websites of ten major nationwide German news outlets were included. The resulting sample is heterogeneous in terms of political orientation (ranging from moderate left-wing taz to liberal-conservative Die Welt) and reporting style and quality (including tabloid Bild, weeklies Zeit, Spiegel, and FOCUS, the online-only news portal T-Online, and traditional broadsheets SZ and FAZ). We scraped all articles that were published on the respective outlets’ websites using the sites’ RSS feeds for front page news and all available subsections (e.g. politics, economy, culture, etc.). After removing bunk items, 697,913 articles were included.

Selection of Group Names

Names of ethnic groups that were included in the analyses were extracted from a list of countries that was reported in the 2018 edition of German Microcensus conducted by the Federal Statistical Office. These countries represent the country of origin of the largest migrant groups in Germany. We then derived the German demonyms and all their grammatical forms for all the included groups. A full list of group names (and demonyms) can be found in Online Appendix II. During this process, we found two ambiguous terms whose impact we studied in a sensitivity analysis (Online Appendix VI).

Measures

Group-Related Stigmatization

With regard to xenophobic outgroup devaluation, the predominant form of stigmatization is to associate outgroups with fear (van der Veer et al., 2011). The polar opposite of xenophobia is xenophilia, the admiration of outgroups. The Stereotype Content Model (Cuddy et al., 2007) suggests fear and admiration are both rooted in (low or high) perceived outgroup warmth. Yet, while admiration also requires high perceived competence, fear seems to be largely unrelated to competence (Cuddy et al., 2007). Despite this structural difference, both emotions seem to be functional equivalents when it comes to their consequences for group relations. Fear leads to the wish for distance from and avoidance of a group and, simultaneously, heightens stereotypical thinking (Ortiz & Harwood, 2007). Admiration, in turn, increases the wish for a close relationship with the admired person or group (Mackie & Smith, 2018) and was found to reduce outgroup stereotyping (e.g. Seger et al., 2017). In this study, we, therefore, measure these two emotions as expressions of xenophobia and xenophilia that represent two poles of a continuum.

Explicit Stigmatization. Detection of explicit stigmatization is relatively straightforward, because it is a direct measurement of textual sentiment surrounding a specific group entity. We opted against using off-the-shelf dictionaries for sentiment analysis which are not tailored to measure fear and admiration. Instead, we applied a semi-supervised approach using the Latent Semantic Scaling (LSS) algorithm by Watanabe (2021). Despite its name, the method does not measure latent emotions in a sentence. Instead, the LSS algorithm depends on a seed dictionary of explicitly judgmental words to define the two poles on a unidimensional scale. These seed words are then used to identify similar words using the Latent Semantic Analysis (LSA) approach. Subsequently, a weight is automatically assigned to each word.

We made two important modifications to this approach. Unlike the original method, we did not only consider adjectives but all words except nouns and proper nouns, based on part-of-speech tagging using the R package spacyr. Second, our seed words were not specified. Instead, we used a data-driven approach to generate the seed dictionary (Haselmayer & Jenny, 2017). Random sentences were selected that contained at least one group name and one emotion word. For the fear category we relied on a word list from the NRC dictionary. For admiration, we used words from a dictionary by Ludwig (2021). 3,000 sentences for each category were then coded by two coders regarding their dominant emotion: fear (present in 280 or 3.5% of sentences), admiration (123 sentences, 1.3%),² other emotions, or no emotion. The full coding scheme can be found in the osf repository for this study.

Afterward, all coded sentences were divided randomly into two sets. The first set was used to develop the seed dictionary. Keyness of each verb, adverb, and adjective was determined using the previously coded sentences, indicating which words are more likely to carry fear and admiration. The top 120 words in terms of keyness for the two emotions were selected and inspected. From there, 14 fear and 34 admiration words were identified (see, Online Appendix III). The final seed dictionary was used to train the LSS model on 161,222 uncoded sentences containing at least one group name. The model, for instance, captured the word “proud” as indicating admiration (as in “I am a proud Moroccan”) and the word “arrested” as indicating fear (as in “the two Poles were arrested”). Both of these words were not included in the seed dictionary. The level of explicit stigmatization involving a group name is quantified as the mean LSS score of all sentences containing that group name. A higher score indicates a sentence has a higher admiration sentiment than fear sentiment and vice versa. By using the seed word approach and conducting ex-post model validation using manually coded sentences (see, Online Appendix VII) the method ensures that only words associated with the two particular emotions are captured, and not general sentiment.

Implicit Stigmatization. Word embeddings can be used to capture the (latent) meaning of a word. The method is based on the linguistic concept of distributional semantics, in which the (latent) meaning of a word can be inferred by its use in language (Z. S. Harris, 1954). So-called word vectors mathematically represent how often words occur in proximity to each other within a text corpus. This helps identify words with similar meanings. For instance, one can study the level of association of an ethnic group label to attributes indicating emotions such as fear and admiration (e.g. criminal, caring) by measuring the relative similarity of their word vectors (for a full list of fear and admiration words, see Online Appendix XII). As a result, we might find that the word vector of a group label has a higher similarity to the word vector of “criminal” but a lower similarity to the word vector of “caring.” However, for another group label this might not be the case. Yet, there might be other words that cannot directly be related to fear or admiration, but are highly associated with both group labels (e.g., suspicious). These words can be interpreted as constituting a conceptual link between the two groups that allows stigmatizing judgments associated with one group to be applied to the other group as well: If “suspicious” groups are typically “criminal,” a group which is merely depicted as “suspicious” might also be “criminal,” even if this is not explicitly mentioned.

Several methods have been suggested to quantify these so-called “word-embedding biases” (for an overview, see, Badilla et al., 2020). These methods have been employed previously to study implicit racial bias in texts (Caliskan et al., 2017; Kroon et al., 2021). In line with this, we trained word embeddings on our corpus of news articles using the gLoVe algorithm (Pennington et al., 2014). Each word occurring in at least 20 articles has a word vector of length 200. Let v_m denote the word vector of the word m and cos(v₁, v₂) denote the cosine of the angle between vectors v₁ and v₂. With the word sets F (fear words) and A (admiration words) in the seed dictionary, we quantified the word embedding bias of a group name g using the normalized association score NAS(g, A, F), proposed by Caliskan et al. (2017). Similar to the LSS score, a higher normalized association score (NAS) indicates a higher level of implicit association with admiration than with fear:

$NAS\left(g,A,F\right)=\frac{mea{n}_{a\in A}cos\left(v_g,v_a\right)-mea{n}_{f\in F}cos\left(v_g,v_f\right)}{s{d}_{x\in A\cup F}cos\left(v_g,v_x\right)}$

In our analyses, we used the NAS score for each group as an indicator of the groups’ implicit stigmatization in the news corpus. Robustness checks of the NAS model are available from Online Appendices IX and X. It is important to acknowledge that this method is not based on the analysis of texts, but on creating a matrix of words used within a whole corpus. Therefore, it cannot assign specific values of implicit stigmatization to single texts, but only to the objects of evaluation as they are represented within the corpus as a whole.

Independent Variables

GDP per capita of each country included in the group name list was extracted from the 2018 World Bank data. For group size, the total number of immigrants from an origin country living in Germany in 2018 was extracted from the 2018 German micro census data. To assess RQ3, we extracted the total number of immigrants from each country who stated their reason for immigration to be refugee, asylum, or international protection reasons. Following a method proposed by Sheafer et al. (2014), cultural distance was quantified using data from the 2018 World Value Survey (WVS). We calculated the Euclidean distance of each country’s population’s position on survival versus self-expression values and traditional versus secular-rational values. Data for Syria, Afghanistan, and Iran were not available. However, our Bayesian modeling approach allows to account for this missingness by the means of imputation. Findings concerning H2a&b are therefore robust against missingness of WVS data (see, Online Appendix XI). For studying H3, the percentage of Muslim population of the different countries was extracted from data provided by Pew Research Center.

Analyses

Since no random sampling was involved in our data collection, we took a Bayesian multilevel modeling approach using the R package brms (Bürkner, 2018). Analyses were conducted at the group-name level. For example, Bosnier, Bosnierin, and Herzegowiner are considered to be different group names, while they are nested within the same country. The effect of clustering within an origin country was adjusted for by entering a varying intercept. The dependent variables (LSS or NAS of a group name)³ were mean-centered to enhance comparability. We entered one independent variable at a time and all independent variables were log transformed. We interpreted the Bayesian models as per Makowski et al. (2019): (1) 89% HDI was used to determine the likelihood of a null effect, and (2) ROPE (full) was used to determine the probability of the absolute value of the effect size being less than 0.1 standard deviation (equivalent to an effect size of Cohen’s D < 0.1).

Results

In order to gain a first overview of the results, Figure 1 displays the LSS and NAS of all investigated groups (r = 0.204). While there are some groups with low LSS and low NAS, e.g. Tunisians, as well as high LSS and high NAS, e.g. French, the plot also demonstrates considerable variance between countries when it comes to the distribution of both scores.

Figure 1. Explicit and implicit scores of country-based group names.

Scatter plot of the relationship between explicit model score (LSS; x-axis) and implicit model score (NAS; y-axis) with group labels as cases. The plot indicates a moderate correlation between the two scores, with some outliers.

Note: Both explicit and implicit scores are mean-centered.

Explaining Explicit Stigmatization

Table 1 shows the results of multilevel Bayesian regression models with LSS being the dependent variable (for full model results, see, Online Appendix IV). Evidence supports H1 and H2a, i.e. groups being associated with wealthy countries or countries with a closer cultural distance to Germany are less likely to be subjected to explicit stigmatization. However, the data do not support H3a: groups that are associated with countries that have a higher Muslim population are not subject to a higher level of explicit stigmatization. Moreover, groups with a larger number of members living in Germany are less likely to be subject to explicit stigmatization (RQ2a). Yet, there is no evidence for a similar relationship with the group size of forced immigrants to Germany as the country of study (RQ3a).

Table 1. Explaining Explicit Stigmatization: Multilevel Bayesian Regression Models.

	Model for testing
	H1	RQ2a	RQ3a	H2a	H3a
Independent variable	B (89% HDI), % ROPE (Full)
Log GDPPC	0.20 (0.05, 0.34), 15.40%
Log Group Size		0.18 (0.01, 0.36), 20.57%
Log Group Size (Forced)			−0.03 (−0.13, 0.06), 85.70%
Log Cultural Distance				−0.18 (−0.34, −0.05), 17.57%
Log Muslim population					−0.09 (−0.17, 0.01), 59.42%
Intercept	−1.82 (−3.11, −0.43), 0.88%	−1.02 (−1.92, −0.09), 2.30%	0.02 (−0.27, 0.29), 42.23%	0.29 (0.00, 0.58), 13.28%	0.16 (−0.10, 0.46), 30.38%
Bayesian R^2	0.289	0.296	0.290	0.286	0.291

Note: Convergence was confirmed based on the Gelman – Rubin Convergence-Diagnostic.

Explaining Implicit Stigmatization

Table 2 shows the results of multilevel Bayesian regression models for the NAS scores. The data support H2b and H3b, i.e. groups that are associated with countries that are culturally closer to Germany and have a lower Muslim population are less likely to be implicitly stigmatized. The amount of group members living in Germany does not appear to be associated with implicit stigmatization (RQ2b), but the group size of forced immigrants in Germany appears to be so (RQ3b). Groups that are associated with more wealthy countries are also less implicitly stigmatized (RQ1). In addition, the effect sizes of these four predictors for implicit stigmatization are larger than that for explicit stigmatization. This means the variables under study here are better able to explain implicit than explicit stigmatization of ethnic and religious groups in the news.

Table 2. Explaining Implicit Stigmatization: Multilevel Bayesian Regression Models.

	Model for testing
	RQ1	RQ2b	RQ3b	H2b	H3b
Independent variable	B (89% HDI), % ROPE (Full)
Log GDPPC	0.22 (0.10, 0.34), 4.83%
Log Group Size		−0.03 (−0.19, 0.12), 64.53%
Log Group Size (Forced)			−0.10 (−0.17, −0.03), 50.02%
Log Cultural Distance				−0.18 (−0.30, −0.08), 11.70%
Log Muslim population					−0.13 (−0.20, −0.06), 24.55%
Intercept	−2.07 (−3.21, −1.01), 0.03%	0.16 (−0.64, 1.06), 14.32%	0.18 (−0.03, 0.38), 24.05%	0.29 (0.09, 0.50), 6.93%	0.25 (0.04, 0.45), 12.05%
Bayesian R^2	0.101	0.060	0.081	0.088	0.096

Note: Convergence was confirmed based on the Gelman – Rubin Convergence-Diagnostic.

Discussion

The present study argued that journalists, just like other individuals, can hold implicit negative attitudes toward ethnic or religious groups which they can be aware or unaware of and that might consciously or subconsciously affect their coverage. Our analyses demonstrated how word embedding bias can be employed to empirically assess the amount of group-related implicit stigmatization in journalistic texts. Findings indicate considerable differences between groups in terms of the levels of explicit and implicit stigmatization they are facing. Additionally, we tested which factors relating to ethnic and religious groups can explain these patterns of stigmatization in the news. In line with our theoretical expectations, differences in explicit stigmatization are predicted by group-level factors that have been previously established within social psychology and sociology: We find that groups that are associated with less wealthy countries, and with culturally more distant countries, face more stigmatization, both explicitly and implicitly. At the same time, results indicate that intergroup contact can help mitigate these biases, even in news reporting. Groups that are more prevalent within Germany were portrayed less negatively, at least in an explicit way, in German news coverage. This indicates that the positive effects of intergroup contact might partially outweigh a potential increase in threat perceptions toward larger groups. Our results also suggest that intercultural contact might not just alleviate individual-level explicit prejudices – it appears to affect mediated portrayals of the respective outgroups as well.

Concerning implicit group stigmatization in news texts, the present analyses reveal interesting differences when compared to the observed patterns of explicit stigmatization: While economic status and cultural distance appear to have a similar effect on implicit stigmatization as on explicit stigmatization, we do not find an effect of group size on implicit stigmatization. Instead, the data indicate additional effects related to the share of Muslims living in a country, and to the share of involuntary migrants originating from a country. Both of these patterns are not detectable for explicit stigmatization. In line with psychological research on prejudice, our results indicate that anti-discriminatory moral norms might prevent journalists from explicitly expressing implicitly held prejudices against Muslim groups and refugees (Marotta et al., 2019). This is particularly noteworthy as the racialization of Muslims is strongly linked to debates on immigration on one hand (Yurdakul & Korteweg, 2021) and debates concerning discrimination on the other (Balkenhol & Schramm, 2019). It could be that journalists consciously try to avoid feeding these stereotypes, as they are salient within German anti-racist discourse, while unconsciously reproducing them on an implicit level.

These results are particularly interesting in the European context, where racial bias is harder to detect due to a taboo of mentioning race, which leads to biases being expressed along ethnic rather than racial lines (Gingrich, 2004). A negative framing of economically deprived nations, of course, carries the implication of maintaining global status hierarchies rooted in a colonialist past, and cultural distance can serve to maintain cultural dominance (Goldberg, 2006). In national contexts, in which race as a concept is absent from reporting, while groups are racialized along ethnic and religious lines (Wigger, 2019; Yurdakul & Korteweg, 2021), tracing underlying factors of ethnicist bias can aid in making the “absent presence” of racism visible (Balkenhol & Schramm, 2019).

When turning to the role of journalists in shaping these described patterns, Reese and Shoemaker’s (2016) hierarchy of influences model offers an interesting starting point: While anti-discriminatory norms can affect journalistic practice by means of social, organizational, and individual level norms, we would expect these norms to primarily influence the explicit expressions of prejudice in journalist discourse. Meanwhile, implicitly held attitudes (consciously or unconsciously) seem to affect journalistic practice at the individual level and are difficult to counter, let alone detect, within everyday working routines. Therefore, implicit biases can escape these norms. Since implicit biases appear more resistant to social change (Vuletich & Payne, 2019), future research should focus on how these more persistent biases are reproduced and could be reduced within journalistic practice. For political communication scholarship more broadly, the present results should be read as a reminder to include notions of ethnic and religious stigmatization not only when analyzing political message effects, but also in research on analyzing political communication content and its production.

Previous U.S. based research has argued that the distinction between implicit and explicit racial rhetoric may be obsolete since recent survey experiments indicated their priming effects on the audience to be largely the same (Valentino et al., 2018). This research draws from Mendelberg’s (2001) distinction between messages explicitly referring to group labels and messages only using racially connoted associations. However, these associations have to be learned by audience members before their usage can evoke priming effects (see, van Klingeren et al., 2015). By demonstrating that news texts do not only convey explicit derogations of ethnic and religious groups, but may also associate groups with implicit stigma, we offer one explanation for how these patterns might be learned. Thus, our findings should not be read as countering the implicit/explicit model of racial priming (Mendelberg, 2001). Instead, they help understand how implicit priming may occur at all.

Limitations and Future Research

Naturally, this study does not come without limitations. According to the implicit/explicit model of racial priming (Mendelberg, 2001) visual cues contribute strongly to implicit stigmatization. As we base our analyses purely on textual data we might miss out on additional persuasive cues contributing to activation of racial stereotypes. Therefore, future research should interlock visual and textual data for a more holistic approach.

Moreover, our group-level analyses do not disentangle the impact of specific articles, news outlets, or time periods that might have contained extra-ordinarily strong explicit or implicit stigmatization of a specific group. Since we used an article corpus scraped in 2017 and 2018, our data primarily reflect the patterns of stigmatization that occurred in German news texts in this exact period. Yet, our analyses do not focus on the (certainly somewhat volatile) representation of specific groups but on the detection of structural predictors of the levels of implicit and explicit stigmatization that occur with regard to a large set of groups. These deep-rooted cultural patterns of group-based stigmatization should be rather stable, even if one group or another received more negative or positive attention within a certain period of study. Nonetheless, future research should aim at controlling influences of cyclically or otherwise varying stigmatization over time.

Another important question in the context of our analyses is what constitutes explicit stigma in our models. We conducted a qualitative assessment of sentences that the LSS model indicated to contain negative valence. While we found many comprehensible and convincing classifications, we also saw that words such as “assassinated,” “worried,” or “anxious” carried negative valence for the classifier. This leads to the question of how explicit stigmatization is defined. The key to this seems to lie in the communication of agency: the perception of choice and control of the stigmatized group to avoid being “assassinated,” “worried,” or “anxious” (Smith, 2007). Future research will have to refine the methodologies applied here in order to assess agency as it is represented in texts.

Finally, it is difficult to validate our measurement of implicit stigmatization. Word embedding biases are corpus-based measures which make validation additionally difficult. A proper human validation would need raters to read the entire corpus of 697,913 articles and point out what racial biases they have learned from the corpus. We deem this impractical, an issue that has previously been faced by other studies using word embedding biases that did not employ human validation (e.g., Kroon et al., 2021; Sales et al., 2019). Future research should therefore try to develop ways of validating word embedding bias methods using a well-defined causal conjecture.

Conclusion

In this study, we combined automated measures for explicit and implicit stigmatization of groups within news reporting, using latent semantic scaling and word embedding bias respectively. Despite the limitations discussed above, our results show the promise of combining both measures. Moreover, our analyses demonstrate that by linking media content analysis to sociological and social-psychological intergroup theory, avenues to overcome the reproduction of stigmatization in the news become visible. Overall, the main mechanisms that influence hostility toward outgroups – socio-economic status and cultural difference – are also affecting stigmatization in the news. At the same time, intergroup contact seems to alleviate explicit, but not implicit, stigmatization, while anti-Muslim bias and bias against refugees can only be detected using an implicit measure. Large socio-cultural factors affect the tone of coverage overall, while social norms against stigmatization of vulnerable groups and interpersonal contact lower explicit, but not implicit bias.

Sensitizing journalists for the prevalence of these patterns might help to inoculate them against their reproduction. To inform the development of counter-measures, future research should explore the prevalence of implicit racial bias within the attitudinal structures of journalists and other professional political communicators and try to establish causal links between the resulting findings and patterns of stigmatization in political discourse.

Open scholarship

This article has earned the Center for Open Science badges for Open Materials and Preregistered. The materials are openly accessible at https://osf.io/hncx4/.

Supplementary material

Supplemental data for this article can be accessed on the publisher’s website at https://doi.org/10.1080/10584609.2023.2193146

Disclosure statement

No potential conflict of interest was reported by the authors.

data availability

The data will be made available upon request by contacting the corresponding author

Additional information

Funding

This research was supported by the German Federal Ministry for Family Affairs, Senior Citizens, Women and Youth (BMFSFJ) through a grant to the “Research Association Discrimination and Racism” (FoDiRa) of the DeZIM-Research Community (German Center for Integration and Migration Research) and a grant to DeZIM’s National Discrimination and Racism Monitor (NaDiRa).

Notes on contributors

Philipp Müller

Philipp Müller (PhD, LMU Munich, 2015) is Senior Lecturer in the Institute for Media and Communication Studies, University of Mannheim, Germany, and Project Director at the Mannheim Center for European Social Research (MZES). His research interests include political communication content, uses, and effects, as well as the social and psychological dynamics of media change and innovation.

Chung-Hong Chan

Chung-hong Chan (PhD U of Hong Kong, 2018) is Senior Researcher in the Department of Computational Social Science, GESIS – Leibniz Institute for the Social Sciences, Cologne, Germany, and External Fellow at the Mannheim Center for European Social Research (MZES), University of Mannheim (Germany). An epidemiologist by training, he is interested in developing new quantitative methods for communication research.

Katharina Ludwig

Katharina Ludwig is a Research Associate, Lecturer and PhD candidate at both the Institute for Media and Communication Studies, University of Mannheim, Germany, as well as the Mannheim Center for European Social Research (MZES). She studies political communication with a focus on polarization, fragmentation, and extremism as well as migration, racism, and discrimination.

Rainer Freudenthaler

Rainer Freudenthaler (PhD U of Mannheim, 2021) is a Postdoc in the Mannheim Center for European Social Research (MZES), University of Mannheim, Germany. His research focus is on traditional, partisan and alternative media online, their contribution to democratic public spheres, and on implicit and explicit biases within news coverage.

Hartmut Wessler

Hartmut Wessler (PhD U of Hamburg, 1998) is Professor of Media and Communication Studies at the University of Mannheim, Germany, and Project Director at the Mannheim Center for European Social Research (MZES). His research focuses on the quality of public debates in the mass media and on social media platforms with a special emphasis on normative theories of democracy and on intercultural comparisons.

Notes

1. The preregistration included additional research questions on the impact of dual-identity groups and the history of constellation-specific intergroup relations. These could not be included in the present manuscript due to space restrictions. Respective rationales, measures and results are presented in Online Appendix I. All Online Appendices and the preregistration document can be accessed via osf: https://osf.io/hncx4.

2. A pretest showed that “admiration” was indistinguishable from “deep respect” during coding. For pragmatic reasons, we therefore had to code admiration and deep respect as reflecting one category, even though in emotions theory they reflect slightly different patterns of positive affect toward a person or a group.

3. These scores were created at the word-level. In the preregistration of this study, we also posed hypotheses related to sentence-level group differences. These analyses have one less level of aggregation than the current word-level. The results of the sentence-level analyses are available in Online Appendices V and VIII. They are corroborating the word-level analyses presented here, despite smaller effect sizes.