Explainable Artificial Intelligence: Objectives, Stakeholders, and Future Research Opportunities

Figures & data

Figure 1. Explanations for AI-based classifications using Grad-CAM (Lapuschkin et al., 2019, p. 3)

Figure 2. Generalized objectives of explainable artificial intelligence

Figure 3. Stakeholder groups of explainable artificial intelligence

Figure 4. Quality criteria for personalized explanations

Table 1. Summary of potential research opportunities and contributions

Research stream	Research question	Research contribution
Behavioral Science	How do AI explanations influence the users’ and mangers’ cognitive perception of the AI?	Knowledge about how explainability may be an important variable in existing theories about human perception of the world and IS artifacts (e.g., affordance theory, mental model theory, sensemaking, UTAUT, and others).
	How do explanations influence employees’ compliance behavior and work practices?	Knowledge on how AI explanations support IT governance.
	How do explanations help to detect bias in managerial decision making?	Knowledge on how a higher degree of AI transparency leads to a better understanding of potentially undesired practices in the organizational offline world, which found their way into the data sets (e.g., when it comes to racial or gender bias).
	Under which circumstances do explanations support or inhibit individual’s trust toward the AI?	Knowledge on how different levels of expertise and personality traits like risk aversion elicit different reactions to AI explanations.
	How can explanations fulfill task-related needs of the different XAI stakeholders?	Knowledge on when and how explanations should be presented to users in order to increase task performance.
	What are adequate metrics to evaluate AI explanations?	Knowledge on the dimensions that are relevant for explanations to be effective; differentiate “good” from “bad” explanations.
	How do explanations influence (de)skilling of employees?	Knowledge on how explanations help to maintain or increase user qualification and self-efficacy regarding AI usage.
Design Science	How can the technical advancements of computer science (e.g., XAI instruments) be integrated with advancements of information systems (e.g., theorizing and categorization of explanations)?	Bring together knowledge and methodical expertise of different disciplines in order to accelerate and improve XAI research across research communities.
	Which features in explanations support the evaluation of an AI’s ethicality and morality?	Derive an understanding of how an AI’s state of ethicality and morality can be evaluated and which information need to be provided via explanations.
	How can the transdisciplinary design of AI explainability across different stakeholders look like?	Conceptualization of a standardized design process for fair, accountable and transparent AI, that take the needs of different stakeholders into account.
	What are design principles on how to build explainable AI systems that allow for a stakeholder- and domain-specific personalization?	Knowledge of technical possibilities to allow for a flexible adaptation of explanations by users (based on their task-specific needs and level of expertise).
	How should mechanisms of push and pull information through explanations look like?	Knowledge on when the system needs to push information on its reasoning or emerging risks, and how the user can be enabled to individually pull explanations (which includes different regulatory needs for explainability of AI according to its criticality).
	How can the analysis of XAI feature usage help to improve the design and hence quality of AI explanations?	Knowledge on how the manual or automatic analysis of AI usage data improve the understanding of the users’ information needs and hence AI explanations.
	How should explanation interfaces in the context of interactive machine learning be designed, in order to improve the AI system based on a users’ feedback to its reasoning?	Improving our understanding on the role of explanations in the context of Human-in-the-loop (HITL) interactions between users and AI.

Lapuschkin, S., Wäldchen, S., Binder, A., Montavon, G., Samek, W., & Müller, K.-R. (2019). Unmasking clever hans predictors and assessing what machines really learn. Nature Communications, 10(1096), 1–8. https://doi.org/https://doi.org/10.1038/s41467-019-08987-4

 PubMedGoogle Scholar