How does language influence the radicalisation process? A systematic review of research exploring online extremist communication and discussion

Figures & data

Figure 1. Where the data in the selected literature was extracted from (N = 16).

Figure 2. The prevalence of the language theme across all selected studies (N = 16).

Figure 3. The prevalence of the overarching themes across the selected research (N = 16).

Table

Author	Overall sample	Method	Study findings
Agarwal and Sureka (2015)	45,344,958 random Twitter posts	Machine learning algorithm to detect extremist tweets and identify linguistic features	Prevalent use of religious terminology and war related themes
			Hateful words and negative emotion correlated with discussion of outgroup members.
Awan (2017)	100 different Facebook pages, associated with extremists	Mixed methodology: Content analysis and a ‘Word Cloud’	Large proportion of the language used was to glorify ISIS and promote hate towards ISIS opposition, inciting violence against them
	50 different Twitter users, associated with ISIS
			Promotion of ‘Them vs Us’ mentality
Badawy and Ferrara (2018)	1,933,016 tweets from the accounts of 25,000 ISIS associated individuals.	A dictionary-based approach used alongside a tweet classifier application, to identify the overall topic and language of each tweet.	Theological issues and violence were the most discussed topics
			Tweets from these accounts spiked during real world events (e.g. Airstrikes by US) occurring in ISIS dominated lands.
			The words ‘Kill’ ‘War’ and ‘fight’ were utilised frequently.
Baugut and Neumann (2020)	44 imprisoned individuals who were either sentenced due to being a member of an illegal extremist group or became radicalised during incarceration.	Semi-structured problem centred interviews and a content analysis of all interview answers.	Individuals were introduced to extremist propaganda due to the language-based recommendation algorithms used by social media sites.
			The extremist recruiters and ‘preachers’ used sympathetic and encouraging language when communicating with the individuals.
Benigni et al. (2017)	119,156 Twitter User accounts of known ISIS members	Adopted a network analytical approach called ‘Iteratuve Vertex Clustering and Classification’ (IVCC).	High volume of language discussing battlefield
			Hashtags often contained strategic objectives of the Ideology (e.g. #Helparakan referring to the state in Burma ISIS aims to expand into).
			Including terms for ISIS in tweets consistently provided strong indication of core membership to the ideology
Bermingham et al. (2009)	13,000 YouTube profiles, who engaged in a radicalisation discussion	Sentiment analysis	Religious language and reference was frequently used by users
	122,011 YouTube comments, posted by the 13,000 profiles.		Female users demonstrated a greater lack of tolerance for other religions then male users, using negative connotations and language to describe opposing religious views.
			20. Based on language, there is no core radicalisation function to online discussion, with it mostly involving religious communication only.
Bouko et al. (2021)	1720 multimodal posts from 7 Facebook profiles associated with radical Jihadism.	Discourse analysis	Ingroup positivity language was prevalent in the propaganda material posted by these users.
			Language involved with outgroup negativity and hostility was highly prevalent
			Intensive language was used to denounce oppression, often including metaphors to show emotional pain and promote sympathy.
Caiani and Parenti (2009)	46 websites and forums dedicated to extreme right-wing Italian organisations.	Content analysis	Bibliographical references were the most prevalent form of communication on these sites.
			Open and inviting language adopted by users and admins of the site, providing their email and home addresses.
			The words ‘enemies’ and ‘allies’ were used prevalently throughout the online discussions.
Gerstendfeld et al. (2003)	157 extremist websites	Content analysis	Positive language about the group, claiming how they are were misunderstood and not racist, was frequently displayed on the sites.
			Majority of the language involved encouragement to ‘know your enemy’ and urge action against perceived injustices towards their group and ideology.
Grover and Mark (2019)	103,722 comments from alternative right-wing discussion forums on the website Reddit.	Use of hate speech detection model ‘HateSonar’	The most frequently used terms on the subreddits were ‘White’ ‘Jews’ ‘alt’ and ‘race’.
			There was a significant correlation between language involving race and negative discourse
			Language highlighting ‘outgroup’ members was frequent, often adopting the terms ‘we’ and ‘they/them’.
			Majority of language included references and terms associated with hate and violence.
Klausen (2015)	154,119 tweets from 59 twitter accounts linked to western-origin freedom fighters currently in Syria.	Content analysis	Tweets predominately contained religious instruction, providing advice on how to follow their ideology
			Language contained discussion of the brotherhood amongst the fighters and how other users can do their part.
			When discussing the enemy of their ideology, hate and derogatory language was used
Koehler (2015)	8 former right-wing extremists.	Problem centred interviews	Language discussing the ideology provided users with encouragement that the movement would eventually become the norm and achieve its goal
			Extremist recruiters use comforting and casual language
Riebe et al. (2018)	57 Facebook posts and 6784 comments on Facebook, discussing the German extreme right-wing organisation ‘Alternative fur Deutschland’.	Content analysis	Majority of posts contained outgroup discrimination, using derogatory language
			Language involving the promotion of fake news and conspiracies against outgroup members was highly prevalent.
Scrivens (2021)	124,058 posts created by 7,154 users on the right wing website ‘Stormfront’	Sentiment analysis	Emotional, hateful language was highly prevalent amongst the material posted by users
			Posts often contained derogatory language towards the groups perceived enemies, mainly targeting the Jewish community.
Torregrosa et al. (2020)	17,000 Tweets from 112 accounts of known extremists related to Islamic State	Linguistic Inquiry Word Count	Religious language were highly prevalent throughout the sample
			War language such as ‘Attack’ ‘Army’ and ‘Solider’’ were some of the most prevalent terminologies in the sample.
Windsor (2020)	Blog of Aqsa Mahmood, an individual who radicalised and became extremist recruiter.	A discourse analysis	Her language included instead positive affirmation about her choice to emigrate, encouraging others.
			Religious references increased throughout the process, with the inclusion of increased solicitation of other females interested in the ideology.

Agarwal, S., & Sureka, A. (2015). Using KNN and SVM based one-class classifier for detecting online radicalization on Twitter. Proceedings of the 11th International Conference on Distributed Computing and Internet Technology, 8956(1), 431–442. https://doi.org/10.1007/978-3-319-14977-6_47

 Google Scholar

Awan, I. (2017). Cyber-Extremism: Isis and the power of social media. Social Science and Public Policy, 54(1), 138–149. https://doi.org/10.1007/s12115-017-0114-0

 Google Scholar

Badawy, A., & Ferrara, E. (2018). The rise of Jihadist propaganda on social networks. Journal of Computational Social Science, 1(1), 453–470. https://doi.org/10.1007/s42001-018-0015-z

 Google Scholar

Baugut, P., & Neumann, K. (2020). Online propaganda use during Islamist radicalization. Information. Communication & Society, 23(11), 1570–1592. https://doi.org/10.1080/1369118X.2019.1594333

 Google Scholar

Benigni, M. C., Joseph, K., & Carley, K. M. (2017). Online extremism and the communities that sustain it: Detecting the ISIS supporting community on Twitter. PLoS ONE, 12(12), 1–23. https://doi.org/10.1371/journal.pone.0181405

 Web of Science ®Google Scholar

Bermingham, A., Conway, M., Mclnerney, L., & O’Hare, N. (2009). Combining social network analysis and sentiment analysis to explore the potential for online radicalisation. Advances in Social Network Analysis and Mining, 1(1), 231–236. https://doi.org/10.1109/ASONAM.2009.31

 Google Scholar

Bouko, C., Naderer, B., Rieger, D., Ostaeyen, P. V., & Voue, P. (2021). Discourse patterns used by extremist Salafists on Facebook: Identifying potential triggers to cognitive biases in radicalized content. Critical Discourse Studies, 19, 1–22. https://doi.org/10.1080/17405904.2021.1879185

 Web of Science ®Google Scholar

Caiani, M., & Parenti, L. (2009). The dark side of the Web: Italian right-wing extremist groups and the internet. South European Society and Politics, 14(3), 273–294. https://doi.org/10.1080/13608740903342491

 Web of Science ®Google Scholar

Gerstendfeld, P. B., Grant, D. R., & Chiang, C. P. (2003). Hate online: A content analysis of extremist internet sites. Analyses of Social Issues and Public Policy, 3(1), 29–44. https://doi.org/10.1111/j.1530-2415.2003.00013.x

 Google Scholar

Grover, T., & Mark, G. (2019). Detecting potential warning behaviors of ideological radicalization in an alt-right subreddit. Thirteenth International AAAI Conference on Web and Social Media, 13(1), 193–204. https://ojs.aaai.org/index.php/ICWSM/article/view/3221

 Google Scholar

Klausen, J. (2015). Tweeting the jihad: Social media networks of western foreign fighters in Syria and Iraq. Studies in Conflict & Terrorism, 38(1), 1–22. https://doi.org/10.1080/1057610X.2014.974948

 Web of Science ®Google Scholar

Koehler, D. (2015). The radical online: Individual radicalisation processes and the role of the internet. Journal For Deradicalisation, 1(1), 116–134. http://journals.sfu.ca/jd/index.php/jd.

 Google Scholar

Riebe, T., Patsch, K., Kaufhold, M. A., Reuter, C., Dachselt, R., & Weber, G. (2018). From conspiracies to insults: A case study of radicalisation in social media discourse. Mensch und Computer 2018: Workshopband. (pp. 595–603). https://dl.gi.de/bitstream/handle/20.500.12116/16795/Beitrag_449_final__a.pdf

 Google Scholar

Scrivens, R. (2021). Exploring radical right-wing posting behaviours online. Deviant Behaviour, 42(11), 1470–1484. https://doi.org/10.1080/01639625.2020.1756391

 Web of Science ®Google Scholar

Torregrosa, J., Thorburn, J., Lara-Cabrera, R., Camacho, D., & Trujillo, H. M. (2020). Linguistic analysis of pro-ISIS users on Twitter. Behavioral Sciences of Terrorism and Political Aggression, 12(3), 171–185. https://doi.org/10.1080/19434472.2019.1651751

 Web of Science ®Google Scholar

Windsor, L. (2020). The language of radicalization: Female internet recruitment to participation in ISIS activities. Terrorism and Political Violence, 32(3), 506–538. https://doi.org/10.1080/09546553.2017.1385457

 Web of Science ®Google Scholar