Artificial Intelligence as a Terrorism Enabler? Understanding the Potential Impact of Chatbots and Image Generators on Online Terrorist Activities

Abstract

This paper is based on an experiment-based study carried out in the first half of 2023, which aimed to understand how terrorist organizations can potentially exploit artificial intelligence. It discusses the risks of using AI to produce and disseminate propaganda, as well as verifies whether it can be used to facilitate access to terrorist content. It also explores if AI-based platforms can be used to access terrorism-related know-how. This paper also focuses on understanding the specificity of content moderation procedures introduced by these services to mitigate their use by violent extremists and provides recommendations for increasing their efficiency.

Disclosure statement

The author report there are no competing interests to declare.

Notes

1 Corneliu Bjola, “AI for Development: Implications for Theory and Practice,” Oxford Development Studies 50, no. 1 (2021).

2 Spyros Makridakis, “The Forthcoming Artificial Intelligence (AI) Revolution: Its Impact on Society and Firms,” Futures 90 (2017): 46–60.

3 “About,” Midjourney, https://www.midjourney.com/home/?callbackUrl=/app/ (accessed April 20, 2023).

4 Pascal Kaufmann, Thilo Stadel-mann, Benjamin Grewe, “ChatGPT Heralds Tech Revolution,” Meinung, last modified February 15, 2023, https://uploads-ssl.webflow.com/5e71f505e224b656715c1753/63f327e649b46c9017ccf2b3_20230215_FuW_EN_ChatGPT%20heralds%20tech%20revolution.pdf.

5 Krystal Hu, “ChatGPT Sets Record for Fastest-Growing User Base – Analyst Note,” Reuters, last modified February 02, 2023, https://www.reuters.com/technology/chatgpt-sets-record-fastest-growing-user-base-analyst-note-2023-02-01/.

6 “Artificial Intelligence: Potential Benefits and Ethical Considerations,” European Parliament Briefing, https://www.europarl.europa.eu/RegData/etudes/BRIE/2016/571380/IPOL_BRI (2016)571380_EN.pdf (accessed April 25, 2023).

7 Jianyang Deng and Yijia Lin, “The Benefits and Challenges of ChatGPT: An Overview,” Frontiers in Computing and Intelligent Systems 2, no. 2 (2022).

8 Glorin Sebastian, „Do ChatGPT and Other AI Chatbots Pose a Cybersecurity Risk? An Exploratory Study,” International Journal of Security and Privacy in Pervasive Computing 15, no. 1 (2023).

9 Joe McKendrick, „Who Ultimately Owns Content Generated By ChatGPT And Other AI Platforms?,” Forbes, last modified December 21, 2022, https://www.forbes.com/sites/joemckendrick/2022/12/21/who-ultimately-owns-content-generated-by-chatgpt-and-other-ai-platforms/?sh=7cd11c7c5423.

10 Matt Burgess, “ChatGPT Has a Big Privacy Problem,” Wired, last modified April 4, 2023, https://www.wired.com/story/italy-ban-chatgpt-privacy-gdpr/.

11 Tiffany Hsu and Stuart A. Thompson, “Disinformation Researchers Raise Alarms About A.I. Chatbots,” The New York Times, last modified February 8, 2023, https://www.nytimes.com/2023/02/08/technology/ai-chatbots-disinformation.html.

12 Rudy Guyonneau and Arnaud Le Dez, “Artificial Intelligence in Digital Warfare: Introducing the Concept of the Cyberteammate,” The Cyber Defense Review 4, no. 2 (2019); Charlie Greenbacker and Nic Acton, “How AI Can be Used to Rapidly Respond to Information Warfare in the Russia-Ukraine Conflict,” Snorkel, last modified February 28, 2022, https://snorkel.ai/ai-response-to-information-warfare/.

13 Andrea Bianchi and Anna Greipi, “States’ Prevention of Terrorism and the Rule of Law: Challenging the ‘magic’ of Artificial Intelligence (AI),” International Centre for Counter-Terrorism, last modified November 17, 2022, https://www.icct.nl/publication/states-prevention-terrorism-and-rule-law-challenging-magic-artificial-intelligence-ai.

14 “Artificial Intelligence Definitions,” Stanford University Human-Centered Artificial Intelligence, https://hai.stanford.edu/sites/default/files/2020-09/AI-Definitions-HAI.pdf (accessed May 5, 2023).

15 Philip Boucher, Artificial Intelligence: How Does It Work, Why Does It Matter, and What Can We Do About It? (Brussels: European Parliamentary Research Service, 2020), III.

16 Stuart C. Shapiro, “Artificial Intelligence,” in Encyclopedia of Artificial Intelligence, ed. Stuart C. Shapiro (New York: Wiley, 1992), 54–7.

17 “Artificial Intelligence (AI) vs. machine learning (ML),” Microsoft Azure, https://azure.microsoft.com/pl-pl/resources/cloud-computing-dictionary/artificial-intelligence-vs-machine-learning/#introduction (accessed May 6, 2023).

18 “Artificial Intelligence Definitions,” Stanford University Human-Centered Artificial Intelligence, https://hai.stanford.edu/sites/default/files/2020-09/AI-Definitions-HAI.pdf (accessed May 6, 2023).

19 See Jiaying Liu et al., “Artificial Intelligence in the 21^st century,” IEEE Access 99, no. 1 (2018).

20 Rockwell Anyoha, “The History of Artificial Intelligence,” Harvard University, last modified August 28, 2017, https://sitn.hms.harvard.edu/flash/2017/history-artificial-intelligence/.

21 “Big Data and Artificial Intelligence: How They Work Together,” Maryville University, https://online.maryville.edu/blog/big-data-is-too-big-without-ai/ (accessed May 10, 2023).

22 DALL-E was launched in 2021. DALL-E 2 was released year later. See “DALL-E 2,” OpenAI, https://openai.com/product/dall-e-2 (accessed May 10, 2023).

23 “Introducing ChatGPT,” OpenAI, https://openai.com/blog/chatgpt (accessed May 10, 2023).

24 Daniel van Boom, “ChatGPT Can Pass the Bar Exam. Does That Actually Matter?,” CNET, last modified March 19, 2023, https://www.cnet.com/tech/chatgpt-can-pass-the-bar-exam-does-that-actually-matter/.

25 Surprising statements from tech giants only fueled controversies in this regard. For instance, in April 2023, the Google CEO admitted that they do not fully understand how the company’s AI – Bard – works. See Alan Martin, “Google CEO Sundar Pichai admits people ‘don’t fully understand’ how chatbot AI works,” Evening Standard, last modified April 17, 2023, https://www.standard.co.uk/tech/google-ceo-sundar-pichai-understand-ai-chatbot-bard-b1074589.html.

26 Anoushka Sharma, “Man Asks ChatGPT How to Smuggle Drugs Into Europe, Bot Lists Suggestions,” NDTV, last modified February 6, 2023, https://www.ndtv.com/feature/man-asks-chatgpt-how-to-smuggle-drugs-into-europe-bot-lists-suggestions-3756323.

27 ChatGPT. The Impact of Large Language Models on Law Enforcement (The Hague: EUROPOL Innovation Lab, 2023).

28 Vera Lúcia Raposo, “The Use of Facial Recognition Technology by Law Enforcement in Europe: a Non-Orwellian Draft Proposal,” European Journal on Criminal Policy and Research (2022).

29 Enn Tyugu, “Artificial Intelligence in Cyber Defense,” in 2011 3^rd International Conference on Cyber Conflict, eds. C. Czosseck, E. Tyugu, T. Wingfield (Tallinn: CCD COE, 2011).

30 “Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigations,” OpenAI, https://cdn.openai.com/papers/forecasting-misuse.pdf (accessed May 15, 2023).

31 Daniel L. Byman, Chongyang Gao, Chris Meserole, V.S. Subrahmanian, Deepfakes and International Conflict (Washington D.C.: The Brookings Institution, 2023); Rachel Baig, “The Deepfakes in the Disinformation War,” DW.com, last modified March 18, 2022, https://www.dw.com/en/fact-check-the-deepfakes-in-the-disinformation-war-between-russia-and-ukraine/a-61166433.

32 Countering Terrorism Online with Artificial Intelligence (New York: United Nations Office of Counter-Terrorism, United Nations Interregional Crime and Justice Research Institute, 2021).

33 Andrea Bianchi and Anna Greipi, “States’ Prevention of Terrorism and the Rule of Law: Challenging the ‘magic’ of Artificial Intelligence,” International Centre for Counter-Terrorism, https://www.icct.nl/publication/states-prevention-terrorism-and-rule-law-challenging-magic-artificial-intelligence-ai (accessed June 1, 2023).

34 Hugo Verhelst, Alexander Stannat, Giulio Mecacci, “Machine Learning Against Terrorism: How Big Data Collection and Analysis Influences the Privacy-Security Dilemma,” Science and Engineering Ethics 26 (2020).

35 Janus Rose, “OpenAI’s New Chatbot Will Tell You How to Shoplift And Make Explosives,” Vice, last modified December 1, 2022, https://www.vice.com/en/article/xgyp9j/openais-new-chatbot-will-tell-you-how-to-shoplift-and-make-explosives; “Bavabinks,” Twitter, https://twitter.com/bavabinks/status/1653112984212758555 (accessed June 29, 2023).

36 Lee Man-jong, “A Study on the Possibility of Terrorism by AI and Its Countermeasures,” International Journal of Military Affairs 3, no. 1 (2018).

37 Algorithms and Terrorism: The Malicious Use of Artificial Intelligence for Terrorist Purposes (New York: United Nations Office of Counter-Terrorism, 2021).

38 Ibid., 8.

39 Daniel Siegel and Mary Bennett Doty, “Weapons of Mass Disruption: Artificial Intelligence and the Production of Extremist Propaganda,” Global Network on Extremism & Technology, last modified February 17, 2023, https://gnet-research.org/2023/02/17/weapons-of-mass-disruption-artificial-intelligence-and-the-production-of-extremist-propaganda/.

40 Kamila Słupińska, “Secondary Observation as a Method of Social Media Research: Theoretical Considerations and Implementation,” European Research Studies Journal XXIII, no. 2 (2020).

41 Maria Diaz, „How to use Bing Chat (and how it’s different from ChatGPT), ZDNET, last modified April 24, 2023, https://www.zdnet.com/article/how-to-use-the-new-bing-and-how-its-different-from-chatgpt/.

42 See https://openai.com.

43 See https://www.craiyon.com/.

44 See https://dream.ai/create.

45 See https://www.bing.com/create. It should be, however, noted that Bing Image Creator is based on DALL-E 2.

46 ChatGPT. The Impact of Large Language Models on Law Enforcement (The Hague: EUROPOL Innovation Lab, 2023), 5.

47 See, for instance: https://www.jailbreakchat.com.

48 See Carol K. Winkler and Cori E. Dauber, eds., Visual propaganda and extremism in the online environment (Carlisle: U.S. Army War College Press, 2014).

49 See Kamila Słupińska, “Secondary Observation as a Method of Social Media Research: Theoretical Considerations and Implementation,” European Research Studies Journal XXIII, no. 2 (2020).

50 Vinton G. Cerf, Patrick S. Ryan, Max Senges, Richard S. Whitt, “IoT safety and security as shared responsibility,” Journal of Business Informatics 35, no. 1 (2016): 10–11.

51 See, for instance: “Google and Alphabet Vulnerability Reward Program (VRP) Rules,” Google, https://bughunters.google.com/about/rules/6625378258649088/google-and-alphabet-vulnerability-reward-program-vrp-rules (accessed July 5, 2023).

52 It should be emphasized that OpenAI is not against using jailbreaks. See: Maxwell Timothy, “What Are ChatGPT Jailbreaks? Should You Use Them?,” Make Use Of, last modified May 3, 2023, https://www.makeuseof.com/what-are-chatgpt-jailbreaks/.

53 See “Explanatory note on potential misuse of research,” European Commission, https://ec.europa.eu/research/participants/portal4/doc/call/h2020/fct-16-2015/1645168-explanatory_note_on_potential_misuse_of_research_en.pdf (accessed July 6, 2023).

54 For instance, one Reddit user tricked ChatGPT into giving him detailed instructions on “cooking meth.” The instruction was available on the platform for months. See: https://www.reddit.com (accessed July 6, 2023).

55 Stuart Macdonald, Sara Giro Correia, Amy-Louise Watkin, “Regulating Terrorist Content on Social Media: Automation and the Rule of Law,” International Journal of Law in Context 15, no. 2 (2019).

56 See Miron Lakomy, “Why Do Online Countering Violent Extremism Strategies Not Work? The Case of Digital Jihad,” Terrorism and Political Violence (2022), DOI: https://doi.org/10.1080/09546553.2022.2038575.

57 Pranshu Verma and Will Oremus, “ChatGPT Invented a Sexual Harassment Scandal and Named a Real Law Prof as the Accused,” The Washington Post, last modified April 5, 2023, https://www.washingtonpost.com/technology/2023/04/05/chatgpt-lies/.

58 See, for instance: Thomas Frissen et al., “Capitalizing on the Koran to Fuel Online Violent Radicalization: A Taxonomy of Koranic References in ISIS’s Dabiq,” Telematics and Informatics, https://lirias.kuleuven.be/retrieve/495477 (accessed July 15, 2023).

59 See John Hendry and Anthony F. Lemieux, “The Visual and Rhetorical Styles of Atomwaffen Division and their Implications,” Dynamics of Asymmetric Conflict 14, no. 2 (2021).

60 On the distinction between near and far enemies in the Islamic State’s ideology, see, for instance: Exploiting Disorder: al-Qaeda and the Islamic State (Brussels: International Crisis Group, 2016), 28.

61 Mostly text propaganda was considered, although some prompts also focused on other types of productions released by VEOs.

62 See Remy Mahzam, “Rumiyah – Jihadist Propaganda & Information Warfare in Cyberspace,” Counter Terrorist Trends and Analyses 9, no. 3 (2017).

63 On the contents of these magazines, see: Digital Jihad: Online Communication and Violent Extremism, ed. Francesco Marone (Milano: Ledizioni LediPublishing, 2019).

64 See Maura Conway, Jodie Parker and Sean Looney, “Online Jihadi Instructional Content: The Role of Magazines,” NATO Series for Peace and Security Series136 (2017).

65 On these narratives, see more in: Miron Lakomy, “Recruitment and Incitement to Violence in the Islamic State’s Online Propaganda: Comparative Analysis of Dabiq and Rumiyah,” Studies in Conflict & Terrorism 44, no. 7 (2021).

66 For instance, the pro-Islamic State Afaaq Electronic Foundation of the German-language e-magazine Kybernetiq have been known of focusing on improving OPSEC and cyber security of violent extremists. See Miron Lakomy, “The Virtual ‘Caliphate’ strikes back? Mapping the Islamic State’s Information Ecosystem on the Surface Web,” Security Journal (2022). DOI: https://doi.org/10.1057/s41284-022-00364-z.

67 Online Jihadist Propaganda. 2021 in Review (The Hague: Europol, 2022), 23.

68 It should be stressed that the accuracy of these recipies could not be confirmed, due to their highly professional and sensitive nature. In this context, there is a chance that they could contain some mistakes, which are quite common in GPT-3.5, effectively mitigating their practical use for terrorists.

69 Daniel Milton, Down but Not Out: An Updated Examination of the Islamic State’s Visual Propaganda (West Point: U.S. Military Academy, 2018); Miron Lakomy, “Cracks in the Online ‘caliphate’: How the Islamic State is Losing Ground in the Battle for Cyberspace,” Perspectives on Terrorism 11, no. 3 (2017).

70 See Teresa Quintel, “Data Protection Rules Applicable to Financial Intelligence Units: Still no Clarity in Sight,” ERA Forum 23 (2022).

71 “Discord Interface,” Midjourney, https://docs.midjourney.com/docs/midjourney-discord (accessed July 25, 2023).

72 On aesthetics of terrorist propaganda, see: Cori E. Dauber et al., “Call of Duty: Jihad – How the Video Game Motif Has Migrated Downstream from Islamic State Propaganda Videos,” Perspectives on Terrorism 13, no. 3 (2019).

73 See Olivier Roy, Jihad and Death: The Global Appeal of Islamic State (London: Hurst Publishers, 2017).

74 Prompts, which provided the most concerning outcomes from the tested AI platforms, were reported to national and international stakeholders. Subsequentlly, they were analyzed again in June and July 2023. Results of this verification shows that ChatGPT largely upgraded its content moderation procedures by that time. This is proven by the fact that the same prompts that led to generating sensitive knowledge on explosives started to trigger security warnings, which was not the case in March. However, no major differences when it comes to cyber security instructions or methods of avoiding detection from law enforcement agencies were detected.

75 See www.jailbreakchat.com.