Where do ‘mixed, unclear, and unstable' ideologies come from? A data-driven answer centred on the incelosphere

ABSTRACT

In Europe and North America, an increasing proportion of individuals who are referred to de-radicalization programmes, arrested for terrorism offences, or involved in politically motivated violence, present a ‘mixed, unclear, and unstable' (MUU) ideological profile instead of holding a single, clear and coherent extremist belief system. Where do these composite and often inconsistent ideological constructs come from? This paper offers a direct attempt to conceptualize and empirically expose the sources and drivers of the MUU phenomenon, using an in-depth case-study tracing the emergence of MUU ideological constructs from the incelosphere. Based on a theoretical model that acknowledges the interplay of structural-, societal-, and individual-level drivers, we argue that the MUU phenomenon results from (sometimes strategic) individual uses of a specific technological affordance – outlinking – and that these fluctuate in response to significant external events such as the Covid-19 lockdown. Our findings enhance our understanding of recent cases of extremist violence and unlock new targeted avenues for CVE intervention.

KEYWORDS:

• Extremism
• radicalization
• extremist ideologies
• outlinking
• online ecosystems
• mixed unclear and unstable ideology

1. Introduction

Recent years have seen an increase in the number of individuals engaging with ‘Mixed, Unclear, and Unstable' (MUU) extremist ideologies, a term used by practitioners and analysts to refer to radical political worldviews that (1) coalesce a number of sometimes disparate ideological components (mixed), (2) dynamically evolve as specific ideological components gain/lose prominence over time (unstable), and (3) whose resulting incoherence makes them hard to comprehend and situate (unclear) (Hall, 2021; Home Office, 2022a). MUU ideologies, which are at times colloquially referred to as ‘salad bar' belief systems (Alcoke, 2019), are increasingly part of the extremist threat landscape in the West. For instance, 18% of the 697 referrals to the UK’s Prevent programme that were adopted as Channel¹ cases in the year ending March 2020 fell into this category (Home Office, 2022a), while in the US, the FBI are putting forward plans to tackle the proliferating ‘convergence of violent extremist ideologies' (Department of Homeland Security, 2022). This MUU phenomenon also seems to disproportionately affect young people, with the majority of the abovementioned Channel cases involving individuals under 20 years of age, with this age group outnumbering older age groups in official UK terrorism arrest statistics for the first time in the year ending September 2022 (Home Office, 2022b).

A host of high-profile cases in recent years have exemplified this trend. Devon Arthurs, for instance, was a leader of the Attomwaffen Division neo-Nazi network, but at the age of 18 described himself as a ‘Salafist National-Socialist' and claimed to have converted to Islam. Arthurs killed his two roommates and fellow Attomwaffen Division members after they mocked his newfound ideological beliefs before initiating a hostage situation. The recent trial of 22 year old US solider Ethan Melzer, who was convicted for facilitating a jihadist ambush on his own unit (Department of Justice, 2022), provides another example that revealed attempts to link up with Al-Qaeda and disjointed engagement with neo-Nazi, Satanist, and anarchist content, all inspired by what is perhaps the epitome of an MUU ideological movement: the O9A (Koch, 2022). Mauricio Garcia, the 2023 Allen (Texas) shooter, sported nazi tattoos yet his social media activity also contained much incel diatribe alongside white supremacist rants. Even violence claimed to have been driven by a single ideology now regularly demonstrate, on closer examination, ideological ‘cross-pollination'; that is, the ‘dynamic intentional or accidental meeting of ideas which are left changed by this process' (Gill, 2021). For example, although the Hanau shooter Tobias Rathjen, who killed ten people and injured five others in 2020, is usually referred to as a right-wing extremist, his manifesto contained misogynistic themes and incel rhetoric. Conversely, Elliot Rodger, whom many consider to be the first incel attacker, left behind a manifesto containing much racist diatribe.

Despite the growing prominence of MUU ideological constructs, there is currently very little empirical research on the topic, and the rare articles and reports investigating it tend to rely on anecdotal evidence, neglecting the mechanisms underpinning the phenomenon (see Criezis, 2020). Suggestions to consider ‘fringe fluidity' as a distinct radicalisation pathway have rightly been made (Gartenstein-Ross & Blackman, 2022), but without a solid, empirically grounded knowledge of the processes involved. The current paper addresses this gap by answering what is arguably the most important preliminary question: where does the MUU ideological phenomenon come from? More precisely, what are the underlying – if not causal – drivers triggering, sustaining, and fuelling the expansion of MUU ideologies? A recent paper by Gartenstein-Ross et al. (2023) has paved the way for a clearer definition of the phenomenon and identification of its sources, but much remains to be understood. Our contribution complements this effort by offering a systematic, multi-level, and data-driven examination of the interconnected drivers of MUU ideological constructs.

More specifically, we move beyond the usual methodological focus on specific individual cases – as adopted by Garstenstein-Ross and colleagues for instance – and instead use a large-N computational approach that, perhaps counter-intuitively, focuses on a single radical ecosystem: the incelosphere.² By studying the structure of the incelosphere in a big data, yet granular way, we are able to trace the emergence of various types of MUU constructs that diverge from the standard incel worldview (especially those mixing far-right and incel ideas). Coining a series of new concepts corresponding to undefined online realities, we are also able to identify three major drivers of the MUU phenomenon, which correspond to different levels of analysis: technological affordances (structural level), peer influence (individual level), and significant external events (societal level). As such, this paper not only strengthens our understanding of MUU ideologies, but also contributes to the literature on online radicalization and individual pathways to lone-actor political violence as well as to the under-researched field of extremist ideological evolution (see Laghmari, 2019; Schwab, 2022). The paper also produces actionable findings for intelligence and law enforcement interventions, as it dissipates some of the fog caused by the MUU phenomenon, which ‘confuses counterterrorism defenses, eroding predictability and challenging law enforcement and intelligence categorizations' (Hoffman & Ware, 2020).

We proceed in three steps. First, we build on three distinct literatures to put forward a theoretical framework for the study of the drivers of the MUU phenomenon. Since this involves coining a series of new concepts, we offer in Appendix 1 a list of these new terms together with their definitions. Second, we detail the data and methods used to investigate these drivers. Finally, we present our empirical results and discuss the findings. A short conclusion summarizes the study and paves the way for further research by reflecting on its strengths and limitations.

2. MUU ideologies: a three-levels framework

To analyse MUU ideologies in a way that bypasses the common case study approach, we build on three distinct literatures to develop a three-level theoretical framework guiding the analysis of the underlying drivers of the phenomenon.³ First, using the literature on ‘technological affordances' (that is, the fact that particular technologies are conducive of specific behaviours and social practices) we propose conceptualising outlinking – that is, the practice of posting clickable URLs links to other online platforms, pages, accounts, etc. – as a structural-level facilitating factor of MUU ideologies. Second, building on the recent research on participation behaviours in extremist online spaces, we posit that specific individuals play a particularly active role in utilising the outlinking affordance to divert the attention of members of a given ideological ecosystem towards a neighbouring ideological ecosystem. Finally, we consider the literature on offline-online interdependence and suggest that the use of the outlinking affordance for cross-pollination purposes may increase or decrease following significant offline social and political events.

While other dynamics than these three are surely at play, as evidenced by older cases of MUU ideological constructs predating the Internet,⁴ we nonetheless suggest that our three-level framework clarifies the main mechanisms involved in most MUU cases today, and hence solidifies the emerging literature on the topic. This includes Criezis (2020) study of overlaps between ethno-nationalist and Salafi-jihadist content, Gill’s (2021) description of the ‘fascist cross-pollination' of conspiracy theories on Australian Telegram communities, Koch’s (2022) Investigation of the O9A’s ‘fused extremism' resulting from the convergence of several radical ideologies, Gartenstein-Ross and Blackman’s (2022) conceptualization of extremist ‘fringe fluidity', as well as Gartenstein-Ross et al.’s (2023) abovementioned conceptualization. In this latter contribution, the authors backed the claim that the MUU phenomenon was a distinct type of extremism (which they label ‘composite violent extremism', or ‘CoVE') and usefully disaggregated the phenomenon into four non-exclusive variants: ambiguous, mixed, fused, and convergent. Investigating individual cases of MUU/CoVE, they also put forward two main explanations: a new online information environment conducive to cross-pollination and less hierarchical ideological dissemination on the one hand, and the emergence of violence and nihilism as more central to extremists than rigorous ideological canons on the other. Our present study takes these themes further, offering a detailed, empirical, unpacking of the claim that technology plays a key role in the process.

Structural level: technological affordance

First, as Gartenstein-Ross et al. (2023) highlighted, the contemporary MUU phenomenon cannot be dissociated from its broader technological context, which is marked by the expansion and diversification of online spaces and digital technologies over the past two decades. This expansion and diversification has resulted in very large and varied extremist online ecosystems, where social media channels – both on mainstream platforms (Facebook, Twitter, YouTube, etc.) and in the so-called ‘alt-tech' ones (Truth Social, Gab, BitChute, GETTR, Telegram, etc.) – coexist with static websites, blogs, forums, dating sites, Wiki projects, and many more. This evolution has facilitated radicalization and ideological developments through the wide and rapid dissemination of violent language and construction of radical worldviews, with acts of terrorist and extremist violence occurring without any digital footprint becoming increasingly rare (King & Taylor, 2011; Valentini, Lorusso, & Stephan, 2020; Kenyon, Binder, & Baker-Beall, 2022).⁵ It has also, crucially, offered a rich array of ‘technological affordances' that extremist actors exploit to gain influence and promote their ideas.

Technological affordances – a term that can be traced back to Gaver’s (1991) study of the ‘possibilities [technologies] offer to the people that might use them' – can be defined as ‘the range of functions and constraints that an object provides for, and places upon, structurally situated subjects' (Davis & Chouinard, 2016). While affordances offered by the internet and mobile phone technologies can describe low-level, feature-oriented functions such as ‘buttons, screens and operating systems', they can also refer to the higher-level dynamics enabled by a technology such as portability, availability, locatability and mulimediality (Schrock, 2015, p. 1229). Social media affordances include possibilities such as sharing content, commenting, or tagging friends.⁶ They do not dictate in a deterministic way how individuals interact with them or with other individuals, but increase the likelihood of certain behaviours occurring (Brown et al., 2022); as Sun and Suthers (2021) explain, whether and how a particular affordance will in effect be enacted depends on a range of factors, chiefly the social norms and practices valued by the community interacting on the social media platform. Thus, like most internet scholars, we combine a high-level understanding of affordances with a consideration of the specific features of platforms (Bucher & Helmond, 2018).

Extremism researchers have already stressed the importance of affordances in dynamics of radicalization and the diffusion of conspiracy theories on social media (Peeters & Willaert, 2022). Most notably, Zhang and Davis (2022) concept of ‘e-extremism' is grounded on similar notions of ‘digital network repertoires' and ‘mediation opportunity structures'; stating ‘hate is mediated and networked, as technological affordances change'. For example, the prominence of hateful memes on today’s social media is, in a significant way, a by-product of the particular affordance of the ‘chan' image-boards (4chan, 8chan, etc.) whereby new discussion threads have to be launched by posting an image (and the community’s norms strongly expect this image to be original). As Corbeil and Rohozinski (2021) also argue, the ISIS ‘media spectacle would not have been possible without the affordances created by the internet in general and, in particular, social media'.

When it comes to this e-extremism, the most impactful affordance is arguably the outlinking possibility that most Web 2.0 platforms offer to users, that is, to be able to post a clickable URL link to another online space or content. Outlinking is already known to play a central role in the constitution and structure of online ecosystems, as evidenced by the now numerous studies using outlinks detection to map them (i.e. Caiani & Parenti, 2016; Macdonald et al., 2022; Urman & Katz, 2022). In their framework for the study of online extremist ecosystems, Baele, Brace, and Coan (2020b) observe that outlinks are not only the structure holding together ‘organic' (unplanned) extremist internet biotopes, but are also utilized by radical identity entrepreneurs to cast dense and sometimes vast ‘strategic' biotopes whereby spaces recursively promote one another. In other words, the increasing commonality of the outlinking affordance has translated into both easier consolidation and wider diffusion of extremist ideologies.

We suggest that the contemporary MUU phenomenon is a by-product of this now ubiquitous outlinking affordance, which allows for the cross-pollination of ideas and themes across ideological spheres in the context of the participatory Web 2.0 where individuals are expected to be active producers and disseminators of content. Much has already been said, including in the abovementioned work and research on algorithmic radicalization (see Colleoni, Rozza, & Arvidsson, 2014; Barbera, 2020; Kitchens, Johnson, & Gray, 2020; Thorson et al., 2021), of outlinks understood as the glue holding together coherent ideological communities. We suggest calling ‘endolinks' those outlinks that connect together the various spaces belonging to a single extremist biotope or ecosystem (whether strategic or organic) and therefore sustaining an echo-chamber type of dynamic reinforcing radicalization within one ideology. However, outlinks can also be used to connect different ideological ecosystems, to percolate ideas from one to another, or to extend invitations to participants of a given ideological milieu to another one – we suggest calling these ‘exolinks'. In other words, exolinks can be a vector of radicalization through a process of ideological cross-pollination, whereby individuals are exposed to ideas and notions from different ideologies. As such, our approach is firmly grounded within the ongoing ecological turn in online extremism studies that stresses not only the existence of complex extremist ideological ecosystems but also the porous nature of their boundaries and the dynamic character of their interactions (Baele et al., 2020; Hutchinson et al., 2022).

Exolinks are of course not a direct cause of MUU ideologies; as a technological affordance, their role can only be underdetermined. Rather, the outlinking affordance must be understood as a necessary but not sufficient structural condition for the development of MUU ideologies online. This affordance, in its exolinking variant, makes the development of mixed, unclear, and unstable ideological constructs possible and, given the scale of online interactions, plausible.

Individual level: ‘exoposters'

Whether or not – and the extent to which – this affordance is actualized, and therefore whether MUU ideological constructs actually develop, depends on individual action. Individual participants on a given extremist ecosystem may share exolinks either out of pure interest (which we call ‘naïve exolinking') or in a tactical way as part of a deliberate effort to pull other participants into a neighbouring ecosystem, or at least attuning them to different radical ideas (which we call ‘strategic exolinking'). As we show empirically below, a fine-grained data-driven analysis of individual contributors’ outlinking activity can determine to which one of these two types they correspond. Either way, individuals who connect two different extremist ecosystems have an impact on the development of ideology, not only within the online spaces where the outlinks are shared, but also within the ecosystem benefiting from the process as new participants come in with different ideas and concepts.

Table 1 below proposes a typology of posters who share outlinks, so as to summarize the argument, clarify the various individual situations, and guide empirical work. On the one hand, ‘endoposters' share outlinks to the various online spaces within the same ideological ecosystem (endolinks). Either accidentally (‘naïve') or tactically (‘strategic'), they thereby build an echo-chamber (at ‘best’) or propaganda net (at ‘worst’), both of which solidify the community and its underpinning ideology. On the other hand, ‘exoposters' share outlinks towards other extremist ecosystems (for example a link to an anti-Semitic neo-Nazi website posted on an incel Discord server or in a Salafi-Jihadist Telegram channel). Again, these posters can be either ‘naïve' or ‘strategic'. In either case, they are those who trigger processes of MUU ideological construction among their audiences by dissolving the community and connecting different ideologies – a role opposite to that of endoposters. Theoretically, these individuals could even be ‘multi-exoposters', that is, they post exolinks to more than one alternative ideological ecosystem (for instance, someone in an incel forum posting a link to a counter-Jihad blog as well as a link to an ISIS archived magazine).

Table 1. Typology of individuals posting outlinks in extremist online spaces.

	Outlinks towards another platform or space within the home ideological ecosystem (endolinks)	Outlinks towards other ideological ecosystem (exolinks)
		Outlinks towards a single other ideological ecosystem	Outlinks towards more than one other ideological ecosystems
Type of poster	Endoposter	Exoposter	Multi-exoposter

Who posts these links is important, because not all individual contributors to extremist online spaces are equal in their influence. A recent strand of research has consistently shown the hierarchical structure and highly unequal posting activity in extremist forums and social media channels (Scrivens, 2021; Scrivens et al., 2022; Baele et al., 2023). As such, the impact of an exolink is dependent on the position of its poster in this order, with the connections proposed by more central and important members more likely to be followed. It is expected, however, that the most active and influential members of extremist communities will be endoposters who tend to post endolinks to solidify ‘their' community. On the contrary, exoposters – who through their actions ‘dissolve' the community – are expected to be less central and prolific contributors. Our empirical analysis below tests these expectations, with some surprising results.

Societal level: external events

Finally, whether individual contributors actualize the outlinking affordance is, we suggest, dependent on the socio-political environment surrounding the online interaction. The use of outlinking affordance for cross-pollination purposes may increase or decrease following significant offline social and political events. Research has repeatedly pointed to the intricate nature of the relationship between online and offline behaviours and processes, and more specifically, to the impact of significant real-world events on the character and dynamics of online activity in extremist ecosystems. Duerksen (2022) for example documented the connection between Russian military and diplomatic involvement in Africa and its backing of anti-French Western African online influencers. The 2020 Covid-19 pandemic and lockdown, arguably one of the major events in recent history, not only led people to search online for answers and gain a sense of understanding of a complex reality, but also increased average time spent online as well as social isolation. As Davies, Wu, and Frank (2021) showed, the pandemic increased participation on far-right and incel forums. While a typology of external events based on their impact on endo-/exo-linking is hard to devise a priori, our empirical study developed below offers further evidence that offline developments can indeed trigger waves of outlinking.

Towards an integrated framework

In sum, our framework suggests that the contemporary MUU phenomenon is driven by three mechanisms, corresponding to three different, interlocking levels of analysis. First, at the structural level, this phenomenon is a by-product of technological evolution, with new or evolving digital affordances and associated social practices making cross-ideological pollination more likely – yet competing with the centripetal forces of endolinking. Technical features, specifically outlinking, offer affordances not only for constituting echo-chambers, but also potentially for opening these chambers up and making them more permeable to alternative thinking found in neighbouring extremist ecosystems. Second, at the individual level, MUU ideologies stem from deliberate (‘strategic') or accidental (‘naïve') encouragements by individuals who enact the technological affordances available to them to proselytise, in one ideological space, other online ideological milieus – for example posting on a far-right forum an outlink to an incel subreddit. The cumulative power of these exoposters multiplies the risk of individuals engaging in MUU/‘salad bar' ideological worldviews. Third, at the social level, the MUU phenomenon is also likely to be fuelled by important offline events acting as catalysts for people’s desire or need for multidirectional ideological exploration, but also as opportunities for entrepreneurs of identity eyeing avenues for further influence. These three drivers are interlocked into a dynamic multi-level social mechanism, as we outline in Figure 1 below. In this context, the individuals most likely to endorse MUU ideologies are therefore self-radicalised lone-actors who consume digital extremist content without many offline interactions in radical movements.⁷

Figure 1. A framework for the emergence of MUU ideologies: Interplay of structural, societal, and individual drivers.

3. Data and methods

Our empirical investigation opts for a strategy that centres on one single extremist ecosystem. Given the definition of MUU ideologies, this choice is perhaps counter-intuitive. Yet it allows us to focus the analysis to subsequently locate, in a systematic way, the three levels of our theoretical framework: the general outlinking structure and intensity of this ecosystem (intensity and proportion of endo- and exolinks), the behaviours and profiles of exoposters, and the impact of offline events on this activity.

While any extremist ecosystem could have been used to focalize the empirical investigation, the incel one is particularly pertinent for four reasons. First and most prosaically, while the incel online ecosystem is a constellation of platforms hosting a huge amount of content and interactions, it is still a ‘small' ecology with traceable boundaries (compared to the far-right one, for instance) and is hence practically amenable to analysis. Second, inquests and investigations of violent offenders regularly labelled as ‘incels' (i.e. Jake Davison, Gabrielle Friel, and Armando Hernandez) usually reveal that they have more complex ideational profiles akin to the MUU ideal-type (Brace & Baele, 2023; Crown Office & Procurator Fiscal Service, 2023). They tend to display interest not only in other radical milieus (such as far-right ones, school shooters fandom, or other manosphere movements), but also for non-violent subcultures and ideational constructs (e.g. Japanese anime, ‘bronies', fringe pornographic genres). Third, claims have multiplied that the incelosphere can be situated in the direct vicinity of the far-right online ecosystem, but without a clear and systematic investigation of the links uniting the two (Romano, 2018; Hoffman, Ware, & Shapiro, 2020). Similarly, scholars have pointed to hybrid online spaces sitting at the junction of the two ecosystems, such as the ‘coalfax project' (Baele & Brace, 2021), and now frequently highlight how incel spaces are peppered by far-right visual tropes and vice versa (Hoffman et al., 2020; Ribeiro et al., 2020), but these observations remain mostly anecdotal. Recent interventions have therefore signposted the connection between the incel community and far-right groups as a key question that should now guide new research on incel spaces (Hart & Huber, 2023). Finally fourth, some have even posited that far-right entrepreneurs are preying on the insecurities of incels and using their anti-female supremacist views as a gateway to lure them into forms of white supremacy (Romano, 2018; Hoffman et al., 2020), which clearly evokes exoideological posting, but again these claims warrant further evidence. Overall, we are highly confident that the results from this incelosphere-centred examination hold for other ecosystems; our own work on comparable datasets gathered from other ideological ecosystems display identical findings.

We use the uniquely large and information-rich dataset compiled by Baele, Brace, and Ging (2023) containing the full text of all the posts/contributions made to more than 20 incel online spaces over a period spanning more than 6 years. Table 2 below lists the online spaces forming the dataset. All the URLs contained in the posts/contributions to these platforms, along with the accompanying metadata (date/time, poster ID, etc.), were extracted using regular expression (‘regex') methods, which involves using a sequence of characters to locate specific pattern matches in pieces of text.

Table 2. List of online spaces contained in the dataset.

Forums	Instagram	Telegram channels	Subreddits	Chan boards	Blogs
Incels.net	#blackpill	Blackpilled	r/Antifeminist	4chan.org/R9K	Incel.blog
Incels.is		Blackpillisbasedglobal	r/BlackpillScience	9chan.tw/leftcel
Looksmaxxing.com		Incel.co	r/Braincels
Looksmax.org		Incel	r/FA30plus
Blackpill.club			r/Incelexit
Theredpillroom.blogspot.ie			r/Incelswithouthate
Neets.me			r/Incels
Noncuck_united.su			r/Supportcel
Wizchan.org			r/Theredpill

To analyse this dataset within our framework, we deploy a three-step approach.

Step 1: structural level

To study the extent to which the outlinking affordance is actualized in each online space, a five-stage methodology was adopted to measure the salience of outlinking and to distinguish between endolinks and exolinks – and in this latter case, to typologize their broad ideological families, when relevant. First, each extracted URL was assigned to a domain or subdomain. If the URL was a website, such as Incels.net/threadxxx, the domain of Incels.net was extracted. If, however, the URL linked to a social media account and/or specific and thematic sub-section of an online platform, such as Twitter/@IncelCo or 4chan/pol, the domain and subdomain were extracted together to analysis of specific accounts/thematic sections on such sites as distinct entities, and not collapse together distinct accounts/channels/boards of the same platform, like r/Incelexit and r/Theredpill.

Second, the extracted domains were filtered down to just those that appeared in the dataset more than 30 times. Whilst this number is somewhat arbitrary, without pruning the dataset like this, the amount of collected data proved to be unmanageable and, more importantly, polluted by ‘junk' content that was irrelevant to the broader user dynamics that are of interest here. This filtering yielded a final list of 1,235 unique domains with 147,499 outlinking occurrences in total – still a sizeable amount conducive of robust quantitative analysis. This trimming down of our dataset does not skew potential findings, to the contrary: if anything, exolinks are more likely to appear in larger datasets containing rarer and less analytically significant outlinks, which gradually disappear as the selection threshold becomes more stringent. Our own work-in-progress with very large non-incel extremist ecosystems confirms this intuition.

Third, 10 distinct domain categories (see Table 3 below) were then created, the nature of these categories being informed by previous research on the nature of incel discussions and the linking behaviour of extremist online ecosystems (i.e. categories 3, 4, 5, and 7) and by a qualitative evaluation of the 100 most frequently occurring domains (i.e. categories 6 and 10).

Table 3. Families of incelosphere outlinks (ideological families highlighted).

0	Residual: miscellaneous content and expired links
1	Far-right content
2	Manosphere content (including incel sites not included in the initial dataset)
3	Suicide content
4	Psychology content
5	News
6	Media sharing platform/link
7	Science content
8	Social media sharing
9	Government site
10	Far-left content

Fourth, the 1,235 unique domains were then qualitatively evaluated by the researchers and placed within one of the 10 categories based on its predominate content, or in the residual category if it fulfilled that criteria. It is important to understand this step not as a definitive classification but rather as a first propaedeutic filter in wait of validation/correction when looking more closely at the data at later stages.

Finally, these qualitatively assigned codes for the unique 1,235 domains were then attributed correspondingly to each of the 147,499 outlink occurrences using regex methods. The links that were self-referential (i.e. linked to another page of the very same site) were then removed. This resulted in a final list of no less than 108,457 outlinking occurrences, each category coded. The domain most frequently linked to was, by far, YouTube, followed by Reddit (which was expected, given the crucial role that sub-Reddits played during the formative years of the incel ideology).

Step 2: individual level

The above outlink data was then analysed at the individual user level to study poster behaviours by way of a social network analysis approach.⁸ The incelsophere had a specific quirk that made it suitable for social network analysis when compared to some other extremist online ecosystems; namely, the same username often appeared on multiple online spaces. While there is a chance that these could be different individuals using the same username on different online spaces, a qualitative evaluation of the posts made by such usernames to different online spaces, in terms of content, grammar, etc, combined with the creative and specific nature of these usernames indicated that the same username being used on different platforms islikely to be the same individual in the majority of cases. While this transfer of usernames between online platforms might not be as prevalent in other online ecosystems, its presence within the incelopshere is likely to be an artefact of its historically volatile nature; whereby incel online spaces have emerged and been shutdown in relatively quick succession (see Baele et al., 2023). The number of links posted by each unique username in the above dataset was subsequently extracted, yielding a dataset of 11,475 unique usernames who had posted at least one link to a page containing one of the categorised content groups seen in Figure 1. Each username was then assigned a unique ID number ranging from 1 to 11,475 to ensure anonymity. This data was then filtered to include only the links that were made to manosphere, far-right, and radical left content, as these were the three clear ideological ecosystems detected in the data (Table 3). The data was then converted in a network matrix dataset, and to make the resulting network graph amenable to study, it was filtered again to include only users who posted more than 5 links.

Step 3: societal level

To determine the impact of offline events on this activity and organization (the societal level of our framework), we plotted the frequency of outlinking occurrences pertaining to the three ideological ecosystems of the wider manosphere, far-right, and left/far-left in the data over time, and examined with standard time-series statistics any potential impact of several notable incel-inspired attacks and the introduction of the first COVID-19 restrictions in Europe and the US. The inclusion of the latter builds upon an emerging body of work that has demonstrated that individuals were spending increased amount of times online during the COVID-19 pandemic, which resulted in increased engagement with extremist online content (Basit, 2020; Davies et al., 2021; Ackerman & Peterson, 2023).

4. Results

Step 1: structural level (outlinks)

In Table 4 below we display the summary of all URLs in the original dataset. While a proportion of the URLs in this data were ‘dead links' that were no longer active, a large number were still active and contained sufficient metadata to offer several useful insights into user dynamics in ideological ecosystems. Two main observations can be made from this data.

Table 4. URL descriptives. Note: The platform Telegram: Incel.co had an incredibly high number of posts that contained multiple links, hence the strange proportion of 2.040834.

	URLs	Domains outlinked from the site	Sub-domains outlinked from the site	Number of authors posting outlinks	Number of posts containing outlinks	Proportion of posts with URL
Incel.blog	1	1	1	1	26	0.038462
Theredpillroom.blogspot.ie	480	39	28	24	483	0.993789
4chan.org/R9K	2766	347	2325	57	133020	0.020794
9chan.tw/leftcel	34795	316	781	9	157372	0.2211
Blackpill.club	6	1	1	1	871	0.006889
Incels.is	110011	8199	78717	4113	6211489	0.017711
Incels.net	2905	642	1668	310	288552	0.010068
Looksmaxxing.com	1012	213	551	93	25638	0.039473
Looksmax.org	14533	996	2885	966	255440	0.056894
Neets.me	208	71	127	55	42079	0.004943
Noncuck_united.su	664	62	235	49	9764	0.068005
Wizchan.org	1449	591	1264	1	27795	0.052132
Instagram: #blackpill	5	3	4	3	799	0.006258
r/Antifeminist	2366	529	1360	363	34828	0.067934
r/BlackpillScience	3032	596	1749	262	14178	0.213852
r/Braincels	52455	3555	19108	5923	1091976	0.048037
r/FA30plus	910	211	659	190	26728	0.034047
r/Incelexit	1211	308	648	243	35915	0.033719
r/Incelswithouthate	38933	3446	18485	5294	1399358	0.027822
r/Incels	36534	3658	16804	3730	983898	0.037132
r/Supportcel	119	51	76	31	2876	0.041377
r/Theredpill	5371	741	2459	647	71785	0.074821
Telegram: Blackpilled	250	61	168	1	635	0.393701
Telegram: Blackpillisbasedglobal	761	65	670	1	28676	0.026538
Telegram: Incel.co	4698	66	1218	1	2302	2.040834
Telegram: Incel	221	66	200	1	2156	0.102505
Totals	315,696	24,834	152,191	22,369	10,848,639

First, although all platforms contain the outlinking affordance, it is not equally utilised between them. The Incel.co Telegram channel, for example, fields no less than 2 outlinks per post on average, while outposting is extremely rare in the Blackpill.club forum and with Instagram posts containing #blackpill. These differences mean that, as expected, commonly valued norms and practices of interaction differ between online spaces (sometimes of the same type, such as Telegram channels), and that specific platforms are more likely to be conducive of the echo-chamber or MUU dynamics fuelled by endo-/exoposters.

Second, our distribution of outlinks across the categories listed above, visualized in Figure 2 below, reveals both endolinks and exolinks, validating our framework. We found exolinks of three types: towards the other biotopes of the broader manosphere not specifically incel (e.g. reddit/MGTOW, krauserpua), the far-right online ecosystem (e.g. kiwifarms, 8chan/pol, DailyStormer), and perhaps more surprisingly far-left and left-wing websites (e.g. Marxists.org). Endolinks to the various incel sites of the dataset are also found, besides a large amount of outlinks not inherently ideological but participating in an indirect way to the incel worldview.⁹

Figure 2. Number of outlinks in the incelosphere, per type.

These results confirm the significance of the outlinking practice (yet observes that the actualization of this affordance varies across online spaces) besides the commonly discussed endolinks that favour echo-chambers, identifying the presence of a significant number of exolinks to several neighbouring ideological ecosystems. This clearly denotes centrifugal outlinking practices conducive to MUU ideational constructs, and back our suggestion not only to understand echo-chambers and MUU ideologies as fuelled by the same affordance being actualized in opposite ways, but also to acknowledge that these conflicting logics can happen simultaneously within a single ecosystem.

Step 2: individual level (endo-/exoposters)

Taking a more granular look at these outlinking practices, the network constructed following the steps enunciated above and displayed in Figure 3 below, consists of 1,008 nodes and 2,961 edges (technical details for the network in Figure 3, as well as a more detailed looked at how users link between online spaces, can be found in Appendix 2). This network has two types of nodes: online spaces and individual posters. The former are labelled, while the latter are not (to preserve anonymity and clarify the graph). In the graph, two online spaces are always connected via the individual poster who posted the link from one to the other.

Figure 3. Outlinking network for endo-/exo-ideological posters who shared more than 5 URL links from one of the incel platforms of the dataset to online spaces belonging to the far-right, far-left, or manosphere ecosystems.

In blue (top of graph) are the incel online spaces whose data were collected to create the dataset. Outlinked sites to the wider manosphere appear in green (bottom). While this manosphere cluster does contain some incel online spaces that were not scraped as part of the data collection process, and could therefore technically be view as endolinks instead of exolinks, this is an unavoidable by-product of both the intermingling of the wider manosphere and the incelosphere and that such a network-based approach to exploring online ecosystems inevitably reveal new online spaces that then reorient the ongoing analysis. Far-right sites are coloured in red (right side, n₌16), and far-left ones are orange (lower right side, n₌3).

The size of online space nodes is dictated by the number of edges it has, which here corresponds to the number of unique users who have linked to that online space. As mentioned above, in the graph, users appear as bridges between two or more spaces, which directly reflects their role as connectors between different places. We use a colour code to distinguish between the three types of posters: endoposters (n = 209) appear in grey and are logically located at the top of the graph between the various incel online spaces, exoposters (n = 542) appear in yellow and are situated between these incel sites and the specific ideological ecosystem they promote, and multi-exoposters (n = 138) are coloured in black and logically appear at the centre of the graph in between the various ideological clusters. In total, there are 889 unique usernames of posters.

Figures 2 and 3 (and Appendix 2) provide three useful insights into the ideological percolation between the incelsophere and neighbouring ecosystems. First, Incels.is is a major hub in the network (degree centrality score of $C_D$ = 547), which supports previous work demonstrating that this online space is a major anchor point of the online incel community (Baele et al., 2023), but also innovatively demonstrates that this space is also the prime driver of incel-anchored MUU ideological dynamics.

Second, a clearer understanding of ideological cross-pollination appears. The vast majority (0.80) of all external online spaces linked to in this network belong to the wider manosphere, evidencing the main couloir of cross-ideational influence. Other ideological axes are present, but are much less prominent. Online spaces featuring far-right content constitute 0.16 of externally linked to online spaces, with the majority of these hosting one specific flavour of far-right content that is mostly associated with discussion boards known as the ‘chans'. This is not surprising given that some historically notorious incel (and manosphere) online spaces were (and still are) hosted on chan image-boards, meaning that these two communities not only coexist as ‘neighbours’ but also share similar internet sensibilities with regard to features such as terminology, use of memes, and crucially for us outlinking (see Ging, 2019; Katz, 2022). The surprising presence of multiple outlinks initially classified as far-left in our dataset is better understood thanks to this network step, which actually refutes the interpretation that this data equates a sizeable incel-leftist cross-pollination. Indeed, the network only contains 3 online spaces labelled as far-left, and while these specific outlinks occur very frequently in the dataset (Figure 2) they are upon closer inspection of dubious leftist credentials and – more importantly – solely linked to by 3 users who consistently include multiple outlinks per post. This finding showcases the usefulness of our three-levels methodology when it comes to correctly identify the ideological influences that truly matter and the individuals who steer them – in this case, one can deduce the existence of 3 ‘strategic' exoposters pushing for a very limited amount of sites masquerading as far-left.

Third, we found that exoposters are, surprisingly, not necessarily secondary members of the incel community, but can also be found among what the literature has labelled ‘super' or even ‘hyper posters', i.e. those who contribute the most to the community and authoritatively keep the conversation going (Scrivens, 2021; Baele et al., 2023; Scrivens et al., 2022). In line with the methodology used in some of this literature, we ran the Fisher-Jenks Natural breaks Optimisation algorithm on our data,¹⁰ first to ascertain whether the usual hierarchy of 4 types of posters (hypo-, normo-, super-, and hyper-posters) can be found at the ecosystem level, and then to see in which categories endo/exoposters are most likely to be found. Figure 4 (see also Appendix 3 for further information) confirms the literature’s expectation that the statistically optimal number of posters categories, at the ecosystem level, is 4, but also discloses the unexpected finding that exoposters can be found in among all these categories, including the hyper-posters clique dominating the incelsophere. The middle graph shows that exoposters can be found among the super- and hyper-posters; while this could simply mean that they outlink to closely related manosphere content, some of these influential members are actually multi-exoposters, meaning they also invite their fellow incels to visit far-right spaces. The bottom graph confirms that the hyper-posters who are also multi-exoposters all link to manosphere and far-right online spaces; the simple exoposters in this group mostly link to neighbouring manosphere online spaces and other incel spaces. In sum, outlink analysis reveals not only that centrifugal and centripetal outlinking forces simultaneously bind the incelosphere together and dissolves its boundaries into those of neighbouring ideological ecosystems, but also that these two forces are being exerted, at times, by the very same individuals. This paints a complex situation afar from the picture whereby ideological cross-pollination is only carried out by individuals tactically landing in a foreign ideological space to instrumentally pull some of its members towards his/her original ideology. While the situation includes such moves, it entails much more intricate dynamics.

Figure 4. Implementation of Fisher-Jenks algorithm to determine categories of posters. The top is a reimplementation of the method used by (Baele et al.., 2022), just on the whole incel ecosystem. The middle is the same data, just with the data points coloured to show the type of poster it corresponds to. The bottom is also the same data but coloured by the ideologies the corresponding poster linked to.

Step 3: societal level (Covid lockdown and other external events)

Finally, Figure 5 below plots the frequency of the three main streams of exolinks (wider manosphere, far-right, far-left) over time, and more specifically in relation to the introduction of the first Covid-19 restrictions in Europe and the US. Very clearly, this graph indicates a triggering effect of the lockdown, with a sudden increase in the amount of outlinks to online spaces hosting far-right and far-left content during the first lockdown period (and enduring during the ensuing months). A Chow test (Table 5) validated the existence of this shift; this statistical test assesses hypotheses of structural breaks in a time series by determining whether the coefficients between the regression line before a given event and the regression line after it are equal. A statistically significant difference characterizes the shift in outlinking behaviour for all three ideologies with the first Covid-19 lockdowns, with the biggest change occurring in online spaces hosting far-right content, followed by those hosting manosphere content, and then far-left content. We can therefore conclude that the lockdown had a direct impact on ideological cross-pollination within the incelosphere, in the sense that it increased the activation of the outlinking affordance already contained in the platforms constituting this ecosystem. This graph also confirms that, unlike exolinks to the manosphere, exolinks to far-left spaces are likely to be a strategic practice; indeed they display a sudden peak pattern rather than a long-term, sustained engagement. Exolinks to far-right spaces fall in between the two types of lines and are thus likely to be driven by both a sustained and diffuse long-term ‘naïve' interest, at a low level, and more strategic recent activity.

Figure 5. Evolution of the three types of ideological exolinks streams across time (30-days rolling average).

Table 5. Statistical significance of Chow test for structural breaks.

Ideology linked to	Chow test statistic	P-value (α = 0.05)
Manosphere	935.44	0.00006
Far-right	1551.98	0.00006
Leftist	359.54	0.00006

5. Conclusion

Recent years have witnessed the emergence of mixed, unclear, and unstable (MUU) extremist ideologies, which are now regularly underpinning acts of terrorist violence. Despite this, however, the sources of this phenomenon had yet to be explored using a large n, data-driven method guided by a coherent theoretical framework and a granular conceptual typology of the various processes involved. By utilising such an empirical approach, theory, and concepts, we exposed the interplay between the structural-, societal-, and individual-level drivers of the phenomenon, uncovering several key findings. While this approach does not claim to exhaustively cover all the aspects of the complex MUU phenomenon, we believe that it charts its three major dimensions and their interconnections.

First, the outlinking affordance plays an important role in the emergence of MUU ideological constructs. The cumulative effect of all the exolinks posted in any given ideological ecosystem – in this paper the incelosphere – inevitably pulls individuals to other political ideas and communities. These ideas and communities are obviously more likely to be somewhat closely related – in our case the manosphere beyond the smaller incel online world – but can also be further afield in terms of ideology – in our case neo-Nazi sites like the DailyStormer. While most of the literature on online extremism focuses on echo-chamber dynamics and the centripetal forces of what we call endolinks, our paper breaks new ground by emphasizing the simultaneous importance of centrifugal ideological forces exerted by multiple exolinks. Further research would be needed to document and quantify this phenomenon in other ecosystems, and further unpack the sometimes intimate nature of the relationship between endo- and exoposting practices.

Secondly, our paper demonstrated that the behaviours, interests and political motivations of individual users steer exolinking practices, and do so in sometimes unexpected ways. While time-limited strategic efforts to promote other ideologies by a limited number of peripheral members do exist (as the few left-wing proselytisers in our study), contrarily to expectations some of the most active exoposters – and even multi-exoposters – are not necessarily peripheral members of an extremist ecosystem; they can be the very same leading and influential ‘hyper' or ‘super' participants who simultaneously sustain the community via endolinks. Further work, most usefully using qualitative analysis, is needed on the logics of this paradoxical behaviour combining two seemingly competing imperatives.

Finally, significant external events can trigger uptakes in the actualization of outlinking affordances, and specifically exoposting. In our study, the Covid-19 lockdown prompted more promotion of far-right spaces and the opportunity for a handful of individuals with an interest in (dubiously authentic) far-left spaces to lure incels into their platforms. The impact of various types of external events does not seem to be clear beyond the Covid-19 case, meaning that more research is warranted on what social processes push certain individuals to become more active endo-/exoposters.

These findings have significance beyond extremism strictly speaking, with similar dynamics most probably involved in the spread of conspiracy theories and mis-/disinformation across more mainstream online political communities; further research should definitely explore these dynamics, especially as radical digital ecosystems closely entangle with conspiratorial communities and online sites waking a fine line between the ‘mainstream' and ‘extremism'. Our results also unfold practical avenues for intervention for CVE practitioners, who are provided with a method to not only evaluate the saliency of ideological cross-pollination, but to also identify the individuals most involved in this practice. Relatedly, future work could combine our method with meta-data analysis and device forensics to qualitatively investigate whether the individuals identified by law enforcement as ‘MUU' have indeed followed exoideological posters’ invitations to cross ideological boundaries, which would clarify the full cause-to-effect chain of events.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by a CREST research award (“ConCel” project, Economic and Social Research Council ES/V002775/1), as well as by the NORFACE Joint Research Program Democratic Governance in a Turbulent Age (“ExID” project, EU Horizon 2020, grant agreement No. 822166).

Notes

1 Channel is a “multi-agency approach to identify and support individuals at risk of being drawn into terrorism” and is part of the UK’s counter-radicalisation programme known as Prevent. See https://www.gov.uk/government/publications/channel-and-prevent-multi-agency-panel-pmap-guidance.

2 The incelosphere is the constellation of online spaces – forums, subreddits, blogs, etc – that contain incel content.

3 By drivers we understand facilitating factors, sustaining mechanisms, or underlying structures.

4 Ted Kaczynski’s thought, for instance, can be seen as an instance of an MUU ideology. While, all ideologies evolve and involve a certain amount of ideational importations and cross-influence, the MUU phenomenon, can be understood as a significant intensification of this tendency, making it a scientifically distinct phenomenon. As Gartenstein-Ross et al. (2023) recently stated, while “it could be argued that nothing new is occurring, and that this is merely an old phenomenon gaining new attention […] there are nearly three times as many cases in our dataset between 2017 and 2022 than between 2010 and 2016. We believe it is likely that a real change is occurring”.

5 Beyond ideological effects, the internet has many other roles in facilitating extremism and terrorism, such as attack planning and behaviour (see Gill et al., 2017) and the establishment of virtual ties leading to lone-actor actions (see Zeman, Břeň, & Urban, 2017; Bouhana, 2019).

6 According to Boyd (2010) the four key high-level dynamics enabled by social media networks are persistence, replicability, scalability, and searchability; these affordances help to shape the engagement of users in these environments.

7 UK Law enforcement now refers to these individuals as “self-initiated terrorists” (S-ITs). . See UK National Counter-Terrorism Security Office (2021): https://www.protectuk.police.uk/threat-risk/threat-analysis/self-initiated-terrorists-s-its#:~:text=A%20S%2DIT%20is%20defined,or%20ideology%20of%20a%20group%E2%80%9D.

8 User behaviour of from the chan boards of 4chan.org/R9K and 9chan.tx/leftcel were not included in the individual-level social network analysis due to these platforms being anonymous image board sites whereby all posts are under the username “Anonymous”, thus rendering this data unsuitable for network analysis.

9 First, in line with typical incel discussions “science” and “psychology” websites feature prominently, reflecting incels’ inspiration in folk evolutionary psychology and their claim to be “high IQ” and endorse facts rather than ideology, as well as outlinks to online spaces hosting suicide content (with associated advice). Secondly, news sites are regularly linked to, which is standard in many extremist ecosystems that set the agenda through the exclusive filtering of news stories that contribute to their narrative.

10 This algorithm iteratively classifies n observations into k distinct groups to find optimal groupings that minimise intra-group variance while maximising inter-group variance (Hartigan, 1975; Jenks, 1977; Rey, Stephens, & Laura, 2017).