Thinking responsibly about responsible AI and ‘the dark side’ of AI

ABSTRACT

Artificial Intelligence (AI) has been argued to offer a myriad of improvements in how we work and live. The notion of AI comprises a wide-ranging set of technologies that allow individuals and organizations to integrate and analyze data and use that insight to improve or automate decision-making. While most attention has been placed on the positive aspects companies realize by the adoption by the adoption and use of AI, there is a growing concern around the negative and unintended consequences of such technologies. In this special issue we have made a call for research papers that help us explore the dark side of AI use. By adopting a dark side lens, we aimed to expand our understanding of how AI should be implemented in practice, and how to minimize or avoid negative outcomes. In this editorial, we build on the notion of responsible AI, to highlight the different ways in which AI can potentially produce unintended consequences, as well as to suggest alternative paths future IS research can follow to improve our knowledge about how to mitigate such occurrences. We further expand on dark side theorizing in order to uncover hidden assumptions of current literature as well as to propose other prominent themes that can guide future IS research on AI adoption and use.

KEYWORDS:

• Responsible AI
• AI Ethics
• Artificial Intelligence
• Explainable AI
• Dark side

1. Introduction

Technological progress is often measured by increased speed and efficiency and how it can make humans and society faster, better, stronger, and happier (Conboy, 2019). Therefore, information systems (IS) research is dominated by studies that focus on technology’s revolutionary or positive power. However, in more recent years, the IS field is beginning to place its attention on complex and often alarming ways in which the use of IT affects organisational and social life – the dark side of technology and its use. There is a growing, fractious tension between our technological capabilities and the human social structures within which the technologies reside. This tension questions the true extent to which technology makes us faster, better, stronger, and happier. Then there are the side effects – technology’s role in making us more unhealthy, sadder, and exhausted (Conboy, 2019; Tarafdar et al., 2015). The contemporary popular press is strewn with examples of technology overuse and addiction (Tarafdar et al., 2020; Turel & Ferguson, 2020), technostress (Tarafdar et al., 2015), security and privacy concerns (D’Arcy et al., 2014; Goel et al., 2017), fake news (Talwar et al., 2019) and the immediate, global reach, and potency of modern weaponry (Small & Jollands, 2006). In particular, artificial intelligence (AI) has attracted attention regarding the dark side implications it can potentially result in. Notable examples include the potential to increase bias and inequality (Akter et al., 2021; Luengo-Oroz et al., 2021), a lack of transparency (Haibe-Kains et al., 2020), and reduced human agency and freedom (Floridi & Cowls, 2019).

Such adverse outcomes of AI use, along with the threat of loss of control or autonomy over superior AI entities, has sparked an ongoing debate about the need to establish a set of principles to effectively govern AI (Barredo Arrieta et al., 2020; Fjeld et al., 2020). In the past several years, every organisation connected to technology policy has proposed a set of guiding principles around AI (Google, 2021; IBM, 2021; OECD, 2021). As a result, there is a growing interest in the notion of responsible AI as a set of propositions or normative declarations about how AI generally ought to be developed, deployed, and governed (Theodorou & Dignum, 2020). These principles that jointly comprise responsible AI are suggested under the logic that they will prevent or minimise the intended and unintended negative effects that AI will introduce to everyday life (Wang et al., 2020). Such concerns, and the subsequent focus on establishing responsible AI principles, have largely emerged from prominent cases where negative effects have been noted. These “AI gone wrong” cases help highlight the important intricacies and aspects when designing and deploying AI (Desouza et al., 2020). However, utilising such a dark-side lens pre-emptively can enable researchers and practitioners to envision the potential negative or unintended effects of AI before they happen and develop appropriate policies and safeguards (Tarafdar et al., 2013).

While the current discourse of AI research has predominantly focused on the potential value and positive impacts of AI (Davenport & Ronanki, 2018), it is perhaps ironic that the field has yet to adopt a dark side metaphor to examine issues that have traditionally been overlooked, ignored, or suppressed (Linstead et al., 2014). The purpose of this special issue is to open up this debate and spark a discussion around the dark side of AI to critically assess the negative and unintended consequences of AI. Issues typically marginalised as outliers, abnormal or deviant, are rarely investigated rigorously. However, such occurrences can allow us to deploy novel technological innovations such as AI in a much more safe, effective, and responsible way (Shneiderman, 2020). Thus, responsible AI principles go hand-in-hand with a thorough understanding of AI through a dark-side lens, as they can be informed by negative or unintended outcomes of AI and operate pre-emptively towards their appearance (Ågerfalk et al., Forthcoming; Clarke, 2019). In addition, the dark-side lens can allow us to envision circumstances or scenarios that have been previously ignored by mainstream research, opening up a more comprehensive and nuanced understanding of the AI phenomenon (Salo et al., 2018).

In this editorial, we adopt a definition of AI, which does not emphasise a specific technology, but instead follows an integrative approach to the key underlying aspects that characterise the AI phenomenon. Specifically, we adopt the definition of Mikalef and Gupta (2021) and define AI as … ” the ability of a system to identify, interpret, make inferences, and learn from data to achieve predetermined organisational and societal goals”. We start by discussing the phenomenon of responsible AI in the next section, its origins, and the current discussion around the notion. The notion of responsible AI is then decomposed and used as an exemplar of potentially damaging issues that can emerge from AI use and as a means of setting up an agenda for future research. Within this agenda, we highlight where the papers of this special issue have contributed and how the dark-side lens that they have adopted helps us advance our knowledge within the IS domain. In sequence, we challenge some key assumptions around the AI phenomenon by utilising a dark-side lens as a way of questioning the dominant paradigm around AI and suggesting a series of research questions. Finally, we conclude with some thoughts around the future of AI within the IS domain, and some key takes from our experience in managing this special issue.

2. What is responsible AI?

The growing interest in AI has also been accompanied by a plethora of cases where negative or unintended consequences have emerged (Fuchs, 2018). A rather infamous example was the Twitter-based chatbot “Tay”, released by Microsoft in 2016. In less than 24 hours, the chatbot that was originally designed to engage in “conversational understanding”, was taken down after users were teaching it to use misogynist, racist, and politically incorrect phrases (Neff & Nagy, 2016). Cases such as these have prompted policy-makers, researchers, and this practitioners to think about how the development and use of AI can follow “responsible” principles (Barredo Arrieta et al., 2020). AI, therefore, introduces new ethical, legal, and governance challenges – including but not limited to – unintended discrimination, biased outcomes, and issues related to the customers’ awareness and knowledge about how AI is involved in decision outcomes (Singapore_Government, 2021). While earlier studies have looked at isolated factors relating to responsible principles, such as elimination of bias (Brighton & Gigerenzer, 2015), explainability of AI outcomes (Gunning et al., 2019), or safety and security (Srivastava et al., 2017), the last few years have seen a move towards a more holistic understanding of what constitutes responsible AI (Dignum, 2019). A recent report published by the Berkman Klein Center for Internet & Society at Harvard University documented 38 such initiatives from different entities and organisations (Fjeld et al., 2020).

Based on the growing consensus on responsible AI, there is an underlying agreement that it constitutes a set of principles that ensure ethical, transparent, and accountable use of AI technologies consistent with user expectations, organisational values, and societal laws and norms (Accenture, 2021). In this regard, the notion of responsible AI captures a diverse set of requirements that need to be met throughout the system’s entire life cycle (European Commission, 2019). As a testament to its importance, responsible AI has even gained traction at the policy-making level, with several countries now defining the fundamental principles that underlie responsible AI and that organisations, private and public, should need to follow (Jobin et al., 2019). For instance, the AI readiness index measuring the degree to which countries are implementing AI technologies now includes a new sub-index, quantifying the degree to which responsible AI principles are adopted (IDRC, 2020). At the same time, there is also a movement towards responsible AI certification from independent agencies (RAII, 2021), as well as best-practice advocacy from large tech companies such as Google (Google, 2022). At the European level, the European Commission recently tasked a High-Level Expert Group on Artificial Intelligence (AI HLEG), an independent expert group, to develop an integrative framework for responsible and trustworthy AI (European Commission, 2019). Through an open consultation process, the expert group developed a guideline that describes the key components that responsible AI includes. Grounded on these principles, the guidelines then provide seven essential requirements that AI systems should meet for responsible and trustworthy AI to be realised, with an eighth of following laws and regulations (European Commission, 2019).

While there are several studies concerning responsible governance of AI applications on a societal and regulatory level (Buiten, 2019; Erdélyi & Goldsmith, 2018), empirical studies at the organisational, business unit, and individual level remain scarce. Despite this lack of empirical research within these contexts, the existing body of work has been able to identify and describe the key dimensions that comprise responsible AI, with work stemming from the perspectives of various stakeholders, including private companies (Benjamins et al., 2019; Google, 2021), academic research (Clarke, 2019; Kumar et al., 2019; Thiebes et al., 2021), consultancy firms (Accenture, 2021; KPMG, 2020; PwC, 2021), institutions (European Commission, 2019; Library of Congress, 2019; Singapore_Government, 2021), non-profit organisations (IEEE, 2021), and even intergovernmental forums and organisations (OECD, 2021; Twomey & Martin, 2020). These dimensions of responsible AI are complementary in nature and are argued to be crucial in minimising any negative or unintended consequences of AI use. Table 1 presents a short description of each dimension with some indicative references.

Table 1. Descriptions of responsible AI dimensions with indicative references

Dimension	Description	References
Fairness	AI systems should enable inclusion and diversity and not lead to discriminatory outcomes.	(European Commission, 2019; IBM, 2021; KPMG, 2020; Singapore_Government, 2021)
Transparency	AI systems should be open and transparent regarding processes and outcomes and facilitate traceability, explainability, and user communication.	(Barredo Arrieta et al., 2020; European Commission, 2019; IBM, 2021; Kim et al., 2020)
Accountability	AI systems should be developed considering the responsibility and accountability of their outcomes with ethics and principles.	(European Commission, 2019; IBM, 2021; Kim et al., 2020; Singapore_Government, 2021; Twomey & Martin, 2020)
Robustness and safety	AI systems should be developed with a preventative approach to risks and in a manner that they behave as intended while minimising unintentional and unexpected harm.	(European Commission, 2019; Gehr et al., 2018)
Data governance	AI systems should ensure that adequate data governance covers the quality and integrity of the data used throughout the entire lifecycle.	(European Commission, 2019; Janssen et al., 2020)
Laws and regulations	AI systems should adhere to the respective laws and regulations that dictate their functioning.	(Buiten, 2019; European Commission, 2019; European Commission, 2021; Desouza et al., 2020)
Human Oversight	AI systems should generate tangible benefits for people and always stay under human control.	(Benjamins et al., 2019)
Societal and Environmental well-being	AI systems should promote sustainability, ecological, and social responsibility and not cause them any harm.	(European Commission, 2019; Kampf & Fashakin, 2021; Nishant et al., 2020)

3. Looking forward: a research agenda

Ensuring that harmful or unintended consequences are minimised or do not occur during the lifespan of AI projects requires a comprehensive understanding of the role of responsible principles during the design, implementation and maintenance of AI applications. The debate about what aspects of AI systems should be emphasised when developing and using them often follows the latest headlines where adverse effects of AI are noted. Some prominent cases include the mishap of Facebook mis-tagging individuals as primates (Wiggers, 2021), Google’s biased hiring AI application which disadvantaged women and Asian candidates (Weston, 2021), as well as Amazon’s hiring AI application that also ingrained biases against women (Jeffrey, 2018). Such cases set a precedent for paying careful attention to how data is tagged and how this is done to ensure non-bias, as well as how the training of AI algorithms might also influence outcomes. As a result, fairness has recently emerged as a key concern for managers and data scientists (Osoba & Welser, 2017).

Nevertheless, if we are to take a holistic view of AI and the potential unintended consequences it can introduce, it is important to examine how each principle for responsible use can lead to negative effects if left unaccounted for. This agenda highlights some key issues that emerge due to the lack of attention to these principles. Examining how negative effects emerge and how they unfold during the design and deployment of AI applications can aid research and practice in understanding how the value from such investments emanates. Furthermore, it is crucial that IS research effectively translates these high-level principles into concrete governance practices that shape AI governance practices. Table 2 presents some example issues that emerge due to non-consideration of key principles and how research can explore these issues. We do not aim to create an exhaustive list of topics that researchers could focus on through this research agenda. Instead, we emphasise how adopting a dark side perspective can open up new avenues of thinking of phenomena and potential solutions.

Table 2. Examples of issues and emerging research questions

Principle	Issues	Emerging Research Questions
Fairness	Biased outcomes that discriminate against specific population groups The exploitation of consumer biases Homogenisation of prices and collusion in markets Inaccessibility to functions due to negligence in design	What role does AI have in uncovering human biases, and what effect does it have in organisational structures? What processes need to be established to minimise human bias in AI applications? How should organisations report responsible AI practices to their customers? How can ethical frameworks inform AI-based business models? What is the role of organisational composition and structure in ensuring fairness in AI outcomes? How do AI affordances change the design requirements of interfaces and user experience?
Transparency	Lack of formalised procedures for explainable AI Inadequate communication of potential risks and limitations of AI systems Limited traceability of erroneous AI-decisions	What is the role of visualisation and design in communicating AI explanations to users? How should explainable AI be communicated to different stakeholders? What is the impact of transparency on use and trust formation in AI systems? How does the communication of risks and limitations influence the adoption and use of AI applications? What effect does embedding traceability markers in AI applications have on team dynamics?
Accountability	Unclear responsibility for outcomes Lack of responding mechanisms to unintended AI outcomes “No-fault” policies becoming the norm in cases of AI-induced harm	How can organisations assign accountability anchors to AI applications? How does AI governance incorporate aspects of responsibility? What auditability mechanisms need to be enforced, and for what types of AI applications? How do accountability principles impact the design and deployment of AI systems? How should AI systems be tested to ensure no harm?
Robustness and Safety	AI decisions placing human life at risk (e.g., autonomous vehicle dilemmas) Data poisoning or model leakage Malicious use of AI (e.g., targeted disinformation campaigns, non-state weaponised consumer drones, AI terrorism) Irreproducible outcomes/decisions	How should ethical dilemmas be resolved when designing and implementing AI agents? How can malicious applications of AI be detected or prevented? What types of early-detection signals can be used to identify malicious uses of AI? How can organisations reproduce AI outcomes on constantly changing algorithms? How should organisations balance transparency of AI with sufficient safety and privacy
Data Governance	Data privacy breaches Socially constructed biases in data Unauthorised access to sensitive data	How should AI security policies be developed? How can organisations ensure employee compliance with AI security policies? What processes need to be established in data governance to minimise data biasing? What are the consequences of finding biases in data used to train AI, and how do organisations resolve these?
Laws and Regulations	Market disruption (e.g., High-frequency algorithmic trading) Job automation (e.g., displacement of many human jobs)	What changes in legal frameworks need to be introduced in the age of AI? What actions should policy-makers prepare for in light of AI-induced disruptions? What public investments need to be made to counter AI disruptions? How do governments deal with job displacements as a result of AI?
Human Oversight	AI-induced misinformation or manipulation Limited human autonomy Increased inequality or inaccessibility to essential resources	What is the impact on humans of AI-induced lack of autonomy? What are the societal implications of AI-based misinformation or manipulation? How can educational curricula be developed so that individuals can detect AI-based manipulation? How does human cognition need to shift during conjoined agency with AI?
Societal Well-being	Promotion of political or societal polarisation Social deskilling Harm to citizens and customers which leads to reputational and financial damage High energy usage and non-environmental-friendly use of AI	How should digital platforms control AI-based content that promotes societal or political polarisation? What new roles open up for deskilled employees? How should AI systems be designed to facilitate circular economies and sustainability targets? Through what applications of AI can organisations promote societal well-being and environmental protection?

3.1. Fairness

Fairness and bias have indeed featured in recent contemporary literature on AI. In this special issue alone, for example, Kordzadeh and Ghasemaghaei (2021) and Marjanovic et al. (2021) cite examples of algorithmic bias and injustice and the implications of both and of the challenges and steps needed in the design of AI to increase fairness and reduce bias. This research to date is undoubtedly welcome.

However, our conceptualisation of the dark side and responsible AI raises new questions to be addressed and perhaps an adjustment in existing research approaches. For example, dark side research tends to be topic-led, so bias is usually studied where bias is the dominant focus of the study. We suggest that bias becomes an almost assumed part of every study, even if it only forms a relatively small part of that study. The reason for this is that so many papers claim to demonstrate a new value-adding and sometimes transformative application of AI. However, we argue that before any new AI technology or new application of technology is to be persuasive, it must discuss and ideally test for any limitations in terms of fairness.

Second, many studies of fairness and bias tend to focus on one particular decision, e.g., one phase of HR recruitment, one round of budget allocation, one set of medical examinations. This illustrates a point discussed below that dark side phenomena are often timeless. We argue that fitness and bias be evaluated across determinable periods and that the temporal variable of the fairness or bias is considered. For example, over a 30-year career, if a minority group suffer from a small degree of AI-induced bias at a late stage in their career, the overall impact on their cumulative earnings over their career may be relatively small, where they lose out on a bonus or promotion in the last one or two years of their career. However, suppose that bias relates to a promotion or job retention in the first year or two. In that case, it may have a detrimental impact on future career development if, for example, they are not afforded training or finances at this formative stage of their career. Years three to thirty of that career may never materialise.

3.2. Transparency

Transparency is a concept that regularly appears in contemporary conversations around AI, particularly about enabling more lucidity on AI outcomes and the procedures around responsibility and accountability (Barredo Arrieta et al., 2020; Dignum, 2019). In this special issue, the study of Giermindl et al. (2021) highlights that one of the risks around AI is that it can impair transparency and accountability as decisions become more opaque and automated, thus being distanced from a human agent. Such phenomena have placed a strong requirement for explainable AI (Barredo Arrieta et al., 2020), whereby the outcomes and data used to derive a decision from AI must be documentable in a format that can be interpreted by different stakeholders (Gunning et al., 2019).

Nevertheless, explainable AI comes with a whole set of challenges, including the difficulty in translating in a structured way the approach through which complex algorithmic networks derive outcomes and the challenge of continuously changing algorithms based on updated data (Barredo Arrieta et al., 2020). At the same time, explainable AI at the user level is subject to an extensive list of contingencies that need to be considered, such as the type of AI outcome at hand, the context, the timeliness, and the criticality of the AI decision. Furthermore, the direct communication of a machine’s output towards a human recipient is subject to human interpretation. Therefore, an exciting area of future research focus is to understand how humans perceive different forms and formats of AI explanations and to what extent they fulfill their requirements (Thomson & Schoenherr, 2020).

Apart from explainable AI, transparency in AI also incorporates aspects such as traceability and communication. This poses a significant research challenge on defining AI governance practices that assign anchors at different parts of AI projects to identify why AI decisions are erroneous. Enabling such ways of documentation throughout the process also facilitates the requirement of auditability. Thus, an interesting area for future study is understanding how traceability requirements shape the formation of AI governance practices within different contexts.

3.3. Accountability

Accountability has been extensively discussed in the literature on AI, particularly in cases where the outcome is high-critical; e.g., healthcare (Lebovitz et al., 2021) and the unclear accountability of autonomous vehicles by referring to the justification of intentions, motives, and rationalities. Especially in light of the reported dark side consequences of AI accountability, principles need to impact the design and deployment of AI systems. In this special issue, the study of Rana et al. (2021) encourages managers to avoid inappropriate outcomes of AI-BA systems. They call for beneficial, explicable, and transparent systems. Highlighting procedural measures on holistic design and deployment, accountability of the underlying AI systems used in the organisations, they discuss how an organisation can assign accountability anchors to AI-integrated business analysis systems. Giermindl et al. (2021) show how people analytics can impair transparency and accountability.

While these articles demonstrate the importance and some options for improving accountability of AI systems, there is a need for IS research to further delve into how accountability as a key principle driving AI technologies influences the development, evaluation and use of applications. For instance, it is important to understand how key stakeholders throughout developing AI systems are responsible and accountable for adverse outcomes. Understanding this distinction and identifying how accountability changes work practices and distribution of responsibilities within the business and IT teams are essential for examining its impacts on AI deployments. It also raises the additional requirement for setting best-practice approaches to ensure that AI systems and their development process are auditable.

Finally, the question of accountability is also inextricably tied with the types of testing performed to ensure that AI applications do not cause harm. One of the key questions when developing and testing AI applications, one of the key questions is how to provide at different development phases that AI applications fulfill a required threshold of safety and how that should be assessed.

3.4. Robustness and safety

Extending on the previous point of accountability, the importance of developing robust and safe AI applications has been one that has been heavily discussed over the last years (Gehr et al., 2018; Russell et al., 2015; Seshia et al., 2016). Such concerns revolve around both intentional and unintentional issues that can emerge during the design and use of systems. Nevertheless, now that AI applications are becoming ubiquitous, understanding how these key concerns translate into functional requirements from the domain experts to the algorithms necessitates a deeper understanding.

One such key issue when developing AI applications concerns the decisional dilemmas AI applications need to address and how they align with the ethical principles we adhere to as a society (Daly et al., 2021). There is, of course, significant variation throughout cultures on what is ethically acceptable and what is not, so the design and implementation of AI technologies must take into account safety and security while following appropriate ethical directions. Therefore, studies could examine how ethical dilemmas translate into different design approaches of AI technologies and optimal ways of designing based on the various contingencies of end-users and contexts (Leidner & Tona, 2021). In other words, values are likely to change in importance to the context (Kazim & Koshiyama, 2021). Thus, the practical instantiation of ethical values and principles will require trade-offs that require further understanding.

Apart from unintentional concerns regarding the robustness and safety of AI, there are also important considerations regarding how to develop and implement security policies around AI. Such security policies span several levels and stages of AI deployment, and their implementation is likely to be subject to different aspects of the socio-technical context. This means that it is important to examine the environment in which such policies are designed and deployed and identify how humans interpret and act on security policies. Finally, such security policies must incorporate preventive measures for cases of data poisoning and ensure that AI models are not leaked but can be used securely to demonstrate the reproducibility of outcomes and decisions.

3.5. Data governance

The principle of data governance concerns all the sequence of the information value chain in AI projects, from creating and collecting data to its end use either by human agents through AI insight or on automated AI-based decisions. In this special issue, the study of Rinta-Kahila et al. (2021) underscores the importance of establishing appropriate governance practices in AI projects to avoid unintended consequences. Specifically, the authors build on a case study of the Australian government’s “Robodebt” program designed to calculate and collect welfare overpayment debts from citizens automatically but ended up causing severe distress to citizens and welfare agency staff. In another study in this special issue, Aaen et al. (2021) follow a longitudinal case study in the Danish healthcare sector, tracking down the emergence, expansion, and eventual collapse of a large-scale data analytics project. The authors highlight the need for governance practices for the functional, stakeholder, and data subsystems dynamics to maintain balance and support project success.

Apart from the results presented in the studies mentioned above, data governance in the age of AI also imposes additional requirements around privacy and data protection, protecting against unauthorised access to data and ensuring that data is of high quality and integral. These aspects around data governance create ample opportunities for further research. For instance, we still have rudimentary knowledge on how data should be treated to ensure that socially constructed biases, inaccuracies, errors, and mistakes are omitted. At the same time, it is important for organisations developing AI not to sacrifice the quality of outcomes due to over-cleaning data. A counter problem concerns the biased tagging of data used to train AI. While early studies have highlighted the risks and adverse effects of doing so (Lloyd, 2018), it is interesting to examine how practices around data tagging are changing and organisations’ approaches to minimise biased AI outcomes.

In the study of Cheng et al. (2021) featured in this special issue, the authors approach the issue of perceived risks in AI-based applications. They show that perceptions of risks can dampen the perceived benefits of AI applications. However, risk perceptions are minimised when organisations establish safeguards for minimising risks and privacy concerns communicated with users. Studies such as this one raise the question about how organisations should develop appropriate data governance practices and advertise and share these with users.

3.6. Laws and regulations

A more responsible AI can be introduced in law and regulations, highlighting the dark side issues. In the comprehensive literature analysis of Kordzadeh and Ghasemaghaei (2021), most papers on algorithmic bias conceptually discussed the ethical, legal, and design implications, while few provided an empirical examination of their claims. In the special issue, we find such empirical evidence on the dark side aspects of AI and big data analysis in organisations, governments, and society.

Marjanovic et al. (2021) sketch out algorithmic justice. They argue that society, particularly automated welfare decisions, is ingrained with systemic algorithmic injustices. The need for an intricate balance of leveraging the potential of AI while ensuring that individuals, particularly vulnerable groups, are not harmed by the effects of the new AI is emphasised. Legislators and policy-makers need to ensure that legal frameworks do not encode injustice for individuals, creating systematic abuse. The article teaches how automated decisions are ingrained with systemic algorithmic injustice that produces dire consequences concerning welfare. Departing from their illuminating case, the authors also call for justice in automated decision-making cases, particularly in relation to societal welfare.

Seeing the market disruptions, predicted consequences of job automation, calls have been made for laws and regulations of AI on a national and global level. In this special issue, Aaen et al. (2021) present a data ecosystem model that calls for a more balanced attitude of data as a strategic asset. Via an intriguing narrative, we learn how a once legally compliant project approved for collecting data on four diseases was re-used and leveraged for broader purposes. The increasing data, often depicted as a non-consumable and non-perishable resource for strategic use, together with extended subsystems functionality and heterogeneous stakeholder goals, instead made the project implode. Subsequently, the whole system was cancelled, including the initially legally compliant functions.

While laws and regulations are undoubtedly important in protecting different stakeholders from negative or unintended consequences of AI, they also impose hurdles when defining restrictions without appropriate ways to deal with such requirements. Hence, policy-making needs to understand how laws and regulations are perceived by the relevant bodies that are subject to comply with them to improve their adherence. In cases where there is too much vagueness, private and public bodies might breach corresponding laws or regulations, or they may even result in non-adoption or use of AI in fear of legal consequences.

3.7. Human oversight

In terms of human oversight, very pertinent questions arise around how we achieve a balance between AI autonomy and ensuring effective and responsible outcomes. Alternatively, we need to ensure that humans retain autonomy while using AI to the most effective extent possible. This topic has brought rise to the notion of conjoined agency and how to balance the strengths of humans and machines in symbiotic relationships (Fügener et al., 2021; Murray et al., 2020).

In addition, our conceptualisation of the dark side questions the current mode of human oversight, which is for AI to conduct day-to-day decisions and actions, and humans maintain oversight by checking any strange, deviant, abnormal occurrences. Our first suggestion is that real damage occurs when a problem is an exception but embedded in many or all AI decisions and actions. By only maintaining oversight of the abnormal, one may ignore AI-induced problems and errors that affect many or all. In addition, given that we have shown deviant and abnormal decisions and actions may not necessarily be a bad thing, it is vital to ensure that processes for human oversight are not automatically designed to eliminate or reduce deviant AI decisions- which is the default position in most studies and applications of AI.

Also, as conjectured below, dark side research usually assumes that the dark side being sought is known. Suppose we extend this to processes for human oversight. In that case, it is logical that this needs to include oversight of known issues such as bias or specified technical errors and scanning for unknown and unprecedented AI decisions and actions that may have unknown negative impacts. In this special issue, Rinta-Kahila et al. (2021) and their discussion of unintended algorithmic consequences provide an illustrative case of this and highlight the need for more research in this area.

3.8. Societal well-being

The issue of AI being developed to promote societal well-being has been featured as a key requirement, particularly from government and regulatory agencies. In this special issue, the paper of Rinta-Kahila et al. (2021) builds on a case study that documents the destructive effects of algorithmic decision-making in public services by causing distress to citizens and employees, as well as financial damage to the welfare agency.

Building on such dark side examples and considering the principle that AI should be developed with societal well-being in mind allows us to examine where adverse effects stem from, what mechanisms sustain or constrain them, and how they can be minimised. There is typically no intent to do so in most prominent examples of AI causing severe societal harm. Therefore, it is important to understand what socio-technical structures and dynamics facilitate such negative outcomes of AI and where more weight needs to be placed to minimise such occurrences. Furthermore, with AI replacing many human and manual tasks, it is necessary to understand what new opportunities emerge for this workforce and how they should be re-trained to fit the AI era.

In addition, there is a growing discussion regarding the monitoring and control of online digital platforms, where AI-based content is used to promote polarisation (Parra et al., 2021). An important area IS research can contribute to is the role of regulation on digital platforms, how it should be implemented, and what consequences it has for the online communities using such platforms. The recent cases that showed that AI algorithms promote higher engagement and polarisation in online communities raise the question of how AI-based recommendations are used without promoting fake news or presenting polarising content.

Finally, with sustainability being a key concern for many government agencies and private organisations, a central question is how AI can be leveraged to support circular economy strategies (Haftor et al., 2021). While there have been many proposals on potential applications, there is a need to explore such approaches in a more systematic way that also considers the footprint of AI technologies in terms of energy usage and resource utilisation.

4. Rethinking the concept of “the dark side”

As part of this special issue, we reviewed key IS literature that refers to the dark side and found an increasing number of papers dealing with dark side issues, but not so much that specifically address the concept of the dark side itself. This contrasts with the studies in other fields (e.g., Linstead et al., 2014). To further thinking on this important topic in the IS domain, we adopted a critical approach to this exercise (c.f. Alvesson & Sandberg, 2011; Myers & Klein, 2011), questioning the core assumptions embedded in studies examining the dark side of technology. We now present some of these assumptions that underpin using the dark side concept in IS.

4.1. The dark side is characterised by the abnormal, the deviant, the outlier

Existing dark side research in IS tends to examine how AI and analytics can lead to exceptional, strange cases. For example, using AI to detect illness may miss rare manifestations of a disease or a deviant; one in a thousand people may use the AI for their own unusual, perverse gain. This emphasis on deviancy is logical, understandable. When one thinks of dark, we often think of that small, dark corner hidden out of sight within which something sinister lurks. This concept underpins conceptualisations of “dark” in other fields. For example, when the field of organisational behaviour (OB) first began to look at the dark side in their context systematically (e.g., bullying, theft, etc.), the first qualifying criterion for something to be considered dark was that it was abnormal or deviant (Linstead et al., 2014). However, we argue that a dark, dystopian side of technology lies in its ability to create deviancy and exceptions and its ability to force Kafka-esque compliance and conformity (e.g., McCabe, 2014). We argue that the self-learning aspects of AI, in particular, can create self-fulfiling myopia of what is acceptable and not, to the extent that diverse, unusual, and potentially transformative ideas and solutions are omitted.

4.2. The dark side is known

The dark side is a term usually used to refer to specific topics in the IS field, e.g., technology overuse and addiction (Tarafdar et al., 2020; Turel & Ferguson, 2020), technostress (Tarafdar et al., 2015), security and privacy concerns (D’Arcy et al., 2014; Goel et al., 2017), and fake news (Talwar et al., 2019). This is also true in the context of analytics and AI, where the dark side term refers to bias, inequality, etc. However, we argue that almost all topics in AI and analytics can, in principle, be the source of dark side issues. Many dark aspects of AI and analytics may be ignored or understudied simply because they do not neatly fit the labels that contemporary researchers associate with the dark side concept.

4.3. The dark side is timeless

An analysis of literature on the dark shows there is often an assumption that time is fixed, or at least that the passing of time does not alter how “dark” an issue is. It is unlikely to be reclassified once something is labelled as dark, e.g., stress, dysfunctional behaviour, or inequality. However, logic would dictate that something dark in one instance may suddenly be reclassified. For example, it is no coincidence that AI is scrutinised regarding bias and inequality (Akter et al., 2021; Luengo-Oroz et al., 2021) given the relatively recent rise of the #MeToo movement and general awareness of gender and race discrimination. However, the discriminatory aspects of technology have been there for many years. Time also plays an immediate key role at the very point that AI or analytics are used, and it is this timing that may determine whether said analytics or AI is, in fact, dark (Conboy et al., 2020; Trocin et al., 2021). For example, we often discuss whether the use of AI for medical diagnosis is or is not ethical and is or is not potentially dark. However, the number of seconds or minutes the surgeon spends assessing the AI recommendation or spot-checking the various AI decisions must in some way affect the ethicality or darkness of its use. Similarly, if the surgeon is at the start or end of a 24-hour work shift when screening or calibrating, the algorithm must also have some bearing. However, these temporal aspects are rarely studied. Table 3 provides a list of existing and updated dark side assumptions, along with examples of emerging research questions.

Table 3. Existing and updated dark side assumptions with examples of emerging research questions

Existing Dark Side Assumption	Updated Dark Side Assumption	Emerging Research Questions
The dark side is characterised by the abnormal, the deviant, the outlier	The abnormal, the deviant, and the outliers may be positive or present positive opportunities rather than something dark to be reduced or eliminated.	How can one evaluate the difference between “good” and “bad” deviants or outliers? How can AI models be developed and trained to accommodate and even seek out these “good” deviants and outliers? How can governance and human oversight mechanisms be designed to accommodate outliers (particularly where specific outliers may require specific additional resource-intensive actions to monitor and control)?
The dark side is known	The dark side may be unknown.	How can the fairness, transparency, accountability, and other features of responsible AI be measured when the issue in question is not known? How can AI models be developed and trained to identify the unknown or accommodate issues that cannot be identified? How can governance and human oversight mechanisms be designed to accommodate the unknown (particularly where the processes required may incur specific additional resource-intensive processes)?
The dark side is timeless	What constitutes dark may change over time, or its negative impacts may materialise at different points in time and have other effects when they do occur.	How can fairness, transparency, accountability, and other features of responsible AI be measured over time? What are the implications of early versus late failures in fairness, bias, transparency, and other features of AI? What impact do temporal complexities such as a rhythm, sequence, or duration of responsibility failures impact overall outcomes? How can AI models be developed and trained to monitor and learn from changes in fairness, bias, and other responsible AI features over time? How can governance and human oversight mechanisms be designed to monitor and address these timing issues?

5. Closing thoughts

In this special issue, we have taken a contrarian view on the AI phenomenon to problematise the field and think through a different perspective of the implications and consequences of introducing AI into work and everyday life. The papers thus featured in this special issue adopt a dark-side lens to inform a more nuanced understanding of how AI is used in practice, its negative or unintended implications, and why and how these emerged. Doing so allows us to focus on the cases where AI has gone wrong and pre-emptively identify cases where AI does not conform with the set requirements. In providing a way forward, we have used the emerging notion of responsible AI as a set of fundamental principles that are important when developing AI applications. We build on the current discourse around responsible AI and develop a set of research questions relevant to the IS domain. While our agenda is by no means exhaustive, it serves as a starting point in thinking about responsible AI more holistically. In doing so, we also highlight the importance of utilising a dark-side lens for theorising and problematising existing recommendations. In addition, the dark-side lens serves as a means of rethinking some hidden assumptions around responsible AI. We argue that while these key principles are important in developing trustworthy and safe AI applications, they are also quite high-level and abstract and do not provide much guidance for practitioners regarding the realities of deploying AI in practice. Our ambition through this special issue and editorial is to encourage future researchers to adopt a more critical approach in IS research on the current debate about responsible AI, which will hopefully result in studies with deep theorising and strong practical implications.

Acknowledgments

This work was supported by the Science Foundation Ireland grant 13/RC/2094_P2, Slovenian Research Agency (research core funding No. P5-0410), and the Wallenberg Foundations, WASP-HS BioMe MMW2019.0112 (2020-2024).

Disclosure statement

No potential conflict of interest was reported by the author(s).