Developing human/AI interactions for chat-based customer services: lessons learned from the Norwegian government

ABSTRACT

Advancements in human/AI interactions led to smartification of public services via the use of chatbots. Here, we present findings from a clinical inquiry research project in a key public service organisation in Norway. In this project, researchers and practitioners worked together to generate insights on the action possibilities offered to human service agents by chatbots and the potential for creating hybrid human/AI service teams. The project sensitised service agents to discover affordances based on their actual practices, rather than on the predefined use of chatbots. The different affordances identified can be useful for practitioners who design and deploy chatbot-based services. The action possibilities afforded by chatbots provide new ways for service agents and chatbots to work as a team addressing citizens’ needs. Drawing from the whole research process, we offer three lessons learned from the Norwegian Government on human/AI partnerships, theory-based interventions, and institutionalised collaborative research that can be useful for researchers that want to engage with practice and organisations that want to evolve their technology use, stimulate innovation, and engage with research.

KEYWORDS:

• Affordances
• augmentation
• automation
• chatbot
• clinical research
• Research Champion
• human–AI partnerships

1. Introduction

Public service organisations experience an increase in digitally mediated requests from citizens (e.g., via webpages, social media, or service apps), which have risen steeply during social distancing. Effectively addressing this surge requires recombining available resources in new ways moving beyond established service delivery models. Chatbots are exemplary artificial intelligence (AI) applications that have been extensively employed to address surges in service requests undertaking communication tasks that used to be performed by humans. However, while certain aspects of good customer service align well with chatbots’ capabilities (e.g., speed, availability, scalability), others correspond better with human competencies (e.g., empathy, judgement, critical assessment). The existence of several unresolved challenges creates opportunities for service tasks to be performed through novel human–AI interactions. With this context in mind, a key public service organisation in Norway (Norwegian Labour and Welfare Administration – NAV) commissioned Information Systems (IS) researchers to work together with in-house practitioners for developing and improving chat-based interaction with citizens.

In Norway, the government promotes the use of AI in public administration aiming to lead the way in developing human-friendly and trustworthy solutions (Norwegian Ministry of Local Government and Modernisation, 2020). NAV launched a chatbot to handle citizen inquiries in the summer of 2018 and kept developing the chatbot service since then. During peak demand, the chatbot has been handling inquiries that correspond to the capacity of 220 service agents. Most inquiries are handled completely by the chatbot but, one out of five get transferred for a live chat with a service agent. Consolidating learnings from chatbot-human interactions is key for finding new ways to recombine available resources to enhance public service delivery.

To explore new forms of human–AI interactions for handling citizen requests we sought to sensitise service agents for discovering new action possibilities based on their actual practices and needs, rather than based on chatbots’ intended use. Sensitising service agents means making them sensitive to what they can do with the chatbots. To achieve this, we designed an intervention, which leverages agents’ experiences, and guides them in a shared and reflective discovery process. When emerging technologies afford new actions, seasoned practitioners can draw from experience for flexibly recombining technologies to generate new practices with transformative effects (Pentland et al., 2022). The importance of affordance discovery for innovation has been identified in prior research (Mesgari & Okoli, 2019) and further explored in this study. The discovery of action possibilities afforded by chatbots to service agents can help them refactor communications by fusing their human competencies with chatbots’ capabilities. In this context, communications’ refactoring means changing existing practices for chat-based service delivery to improve efficiency while retaining the characteristics of good, humane service. Stemming from software engineering, refactoring is a change process that does not alter external behaviours of a system while improving its inner workings (Fowler, 2018). There are significant performance improvements to be reaped by having humans and AI join forces (Raisch & Krakowski, 2021; Wilson & Daugherty, 2018). Nevertheless, most research and practice efforts have focused on traditional chatbot use for task substitution (i.e., chatbots automatically responding to citizen inquiries). The time is right for bringing attention to human augmentation and to the potential of hybrid service teams where humans collaborate closely with machines to perform a task (Raisch & Krakowski, 2021). We, therefore, propose the following Research Questions:

1. What action possibilities are perceived by service agents for chatbot use?

2. How can chatbots and humans be brought together in hybrid service teams?

The paper presents the findings of Clinical Research (Schein, 1987, 2008), performed with the strong involvement of both researchers and practitioners. For over a decade, NAV has been systematically engaging in collaborative research deploying several initiatives including strategic collaborations with universities, funding NAV-initiated and academia-initiated research, and developing employees’ research competence. The organisation has proactively sought collaboration with the researchers’ University by assigning a Research Champion (a person employed by the organisation and physically hosted at the university premises) to establish liaisons, monitor activities in the research and practice sides and sustain good relations. The experience from this project indicates that Research Champion roles can offer a pathway to increase clinical research studies. The project reported in this paper was initiated by NAV by an invitation to the research community for contributing to the improvement of interactions with citizens.

Our work is part of the stream of action-oriented research with practice where knowledge production is driven by organisational needs (Baskerville et al., 2015; Mathiassen & Nielsen, 2008). We drew from affordance theory (Gibson, 1979) and focused on sensitising service agents for discovering new affordances to improve human/AI interactions. What made the affordances truly sensitising was their utility in pointing to directions along which to look, enabling service agents to see beyond the present of human/AI interactions within the organisation. Project activities were organised to help users perceive new action possibilities afforded by chatbots. Service agents were exposed to the ways chatbots are used by others (as reported in publications or as described by other service agents within the same organisation) and were aided to explore further possibilities through discussions with the chatbot development team. The research team framed the sensitising process not as a “benign” data collection activity that precedes the intervention but as an intervention by itself (Schein, 2008). The study findings include six chatbot affordances for service agents that can be useful for practitioners who deploy chat-based services. Our findings provide also an empirically grounded basis for further theorising around human/AI interactions and partnerships (Raisch & Krakowski, 2021) as well as on the potential of AI in reshaping work and service automation (Faraj et al., 2018; Lacity et al., 2021). Drawing from the whole research process, the paper offers three lessons learned on human/AI partnerships, theory-based interventions, and institutionalised collaborative research that can be useful for researchers who want to engage with practice and organisations that want to evolve their technology use.

The remainder of the paper is structured as follows. First, related research on public service and affordances is presented, the method and background information on NAV follows. Then, the study findings are presented and discussed before concluding.

2. Related literature

2.1. Chatbots in public service delivery

Chatbots are increasingly being used for the delivery of public services (Mehr, 2017). They can interact with different citizen groups using natural language, at any time of the day, every day, throughout the year. The growing AI sophistication allows the development of chatbots that are suitable for a wide range of applications. For instance, chatbots have been used for answering citizens’ questions, pointing to relevant documents, or routing citizens to the responsible officers in public services related to social welfare, taxation, municipal services (Aoki, 2020; Androutsopoulou et al., 2019; Mehr, 2017). During the coronavirus crisis, chatbots have proven especially useful in addressing the surge of citizen requests (National Assoc of State Chief Information Officers (NASCIO), 2020). Overall, chatbots have established their role in public administration for information provision to citizens.

Although there are continuous advancements in chatbot training, not all inquiries can be handled by bots. Unlike humans, chatbots are still unable to hold long conversations, to understand which direction a conversation is going, or to match the ability of humans to express empathy in stressful situations (Syvänen & Valentini, 2020). Furthermore, chatbots for public service delivery must meet the needs of diverse population segments including people with disabilities and immigrants that are not native speakers of the local language. We find cases where a public agency attempts to introduce a chatbot service and soon after has to discontinue it due to concerns over its ability to meet public service needs (Benbunan-Fich et al., 2020). Largely, the use of chatbots in public service delivery is challenged by the context-dependency of user inquiries, the wide variety and complexity of administrative services, the differences in expert-language and user-language, as well as the necessity of providing highly reliable answers for all questions (Lommatzsch, 2018). The limitations of chatbots led organisations to combine chatbot service delivery with a live chat takeover by service agents. Chatbots can act as the first line of support handling simple queries and routing more complex queries to service agents.

The potential of joint service delivery by human service agents and chatbots is strong (Baird & Maruping, 2021; Raisch & Krakowski, 2021; Wilson & Daugherty, 2018) and requires more than simply supplementing automated chatbot services with live human-based chats. Chatbots can gradually become “co-workers” within customer service units supporting service agents during live chats. Nevertheless, there is limited research on how chatbots can be brought together with service agents in new types of hybrid service teams. There are benefits to be realised by leveraging chatbots’ capabilities (e.g., speed, availability, scalability) not only for automation but also to complement human competencies (e.g., empathy, judgement, critical assessment), for responding to complex or sensitive inquiries. The symbiosis of humans with chatbots require refactoring communications to use the relative strengths and address the weaknesses of both machines and humans (Gartner, 2017). In software engineering, refactoring is a term used for changing code without altering its external behaviour (Fowler, 2018). The term refactoring, in the context of public service delivery by hybrid chatbot – service agent teams, indicates a direction that preserves citizens’ experience of humane services. The empirical investigation of human-chatbot coordination for joint service delivery requires an understanding of how the capabilities of chatbots relate to service agents’ competences and goals, and the action possibilities they create. In the following section, we introduce the theoretical concept of affordances that provides a lens for relating technological features to specific purposeful user groups.

2.2. Affordances

In IS research, the concept of affordances has been used to convey what a user can do with a technical object, given the user’s capabilities and goals. Affordances are “possibilities for goal-oriented action afforded to specified user groups by technical objects” (Markus & Silver, 2008). The concept has proven useful for studying the interplay between the features of technologies and the characteristics of specific users and use contexts (see relevant literature reviews within IS research: Fromm et al., 2020; Pozzi et al., 2014; Stendal et al., 2016). Affordances are not properties of technologies nor instances of actual use (Markus & Silver, 2008) they are potentialities for purposeful user–technology interaction. These potentialities need to be perceived by users in order to be actualised (Bernhard et al., 2013; Leonardi, 2013; Pozzi et al., 2014). An affordance is perceived when a user becomes aware of an action possibility and may be actualised if the user takes action to realise the possibility perceived.

The benefits from technology relate to the degree to which users perceive and actualise action possibilities (Lehrig et al., 2017). Perceiving action possibilities can strengthen technology use and contribute to better performance. Especially for novel technologies, user explorations, and the identification of new possible ways of using technology are key (Liang et al., 2015). Perceiving what new actions novel technologies can afford leads to new intentions about the use of information systems enabling new work practices (Seidel et al., 2013) and potentially transformative change (Pentland et al., 2022). In the context of chatbots for public service delivery, organisations can find ways to appropriate chatbots innovatively by leveraging their employees’ capabilities to perceive and assess possible uses, that technology designers without domain-specific knowledge and experience may not be able to envision.

Perceiving affordances entails recognising the ways that technology features can be employed for users’ purposes in the specific context of use. Prior research has shown that users may perceive action possibilities by interpreting the symbolic expressions of the technology, by learning about someone else’s technology use, or by sensegiving support from technology specialists that can help users better understand technology (Lehrig et al., 2017; Mesgari & Okoli, 2019). Supporting users’ perception of technology affordances (i.e., helping them discover affordances that may have not been noticed) can accelerate innovation. The importance of affordance discovery and the special role of mediators has been identified in prior research (Mesgari & Okoli, 2019) and has been further explored in this study.

3. Method

In this section, we provide an outline of the method followed. A more detailed presentation of the method is included in the Annex. The study is based on a clinical approach (Schein, 1987, 2008), where the focus of the researchers’ activities at the organisation is to contribute to its improvement. Our findings draw from the practitioners’ experiences and the analysis is jointly performed by researchers and practitioners. The practitioners have been involved in shaping the project during early discussions and throughout its duration. Two of the authors are practitioners working for NAV while the other two are academic researchers. Table 1 provides an overview of key methodological aspects of the study.

Table 1. Overview of key methodological aspects of the study.

	Description	Roles for Researchers and Practitioners
Overall aim & problem formulation	Improvement of interactions with citizens. How can we better leverage chatbot technologies for Human-AI interactions in chat-based customer service? Research focus: service agents´ perspectives.	Invitation to the research community issued by the organisation. Problem narrowed down after a series of discussions between researchers and organisation representatives.
Research design	Clinical research with strong practitioners´ involvement. Affordances as a sensitising concept for theory-based intervention.	Researchers asked to propose the approach and framing that suit the needs of the study. Research design evolved during the project.
Theory building	The study is primarily motivated by the need to generate knowledge useful for organisational development.	Conceptual development through iterations, dialogical reasoning between researchers and practitioners, examining if distinctive aspects are missed. Linkages to related research streams identified by researchers.
Resources	Fully funded by the organisation both for the project and the pre-project problem formulation.	Involvement from both the research and the organisation sides.
Actions & events	Interviews, organisation´s documents analysis, and workshops supplemented with a literature review. Seven workshops: 1. Working on problem formulation leading to the decision to study chatbot use and the handovers from chatbots to service agents 2. Inquiring into the situation leading to an extended focus on service agents’ overall work 3. Working on problem refinement leading to the decisions to investigate service agents’ perceptions for action possibilities when using the chatbot and to conduct a literature review. 4. Engaging with service agents leading to intervention specification: sensitising service agents to discover new action possibilities afforded by chatbots. 5 & 6. Conducting sharing and sensegiving workshops with service agents from different regions, management, and technical team. Sharing interview findings, experiences, and technical understanding leading to initial list of action possibilities based on practices. 7. Presenting and discussing all findings leading to the consolidated list of affordances.	Both researchers and practitioners participated. The research team facilitated and oriented the discussions. Participants prompted to reflect and share experiences on different adaptations in technology configurations and service arrangements, chatbot technical team supported service agents through sensegiving, findings from a literature review on chatbot affordances presented and discussed.
Intervention	Service agents get support to perceive possibilities of using chatbots as assistants for their work. Affordances perceived by service agents feed the technical and management teams for the development of new functionalities and the evolution of service delivery processes.

4. Collaborative research at NAV

The organisation (NAV) has a central role within Norwegian public administration. NAV administers benefit schemes and pensions being responsible for providing services tailored to users’ needs and circumstances, supporting a well-functioning job market, keeping people active and overall, ensuring comprehensive and efficient labour and welfare administration. NAV is proactively promoting research-based knowledge production driven by its needs and uses it for service development and as a basis for labour- and welfare-related decisions, advice and recommendations (Norwegian Labour and Welfare Administration – NAV, 2021). Several research instruments have been developed including strategic collaboration with Universities on specific domains of interest, funding NAV-initiated research and development projects, funding research-initiated projects related to NAV´s knowledge needs, funding “knowledge summaries” on particular topics and developing employees’ research competence through courses and enrolment in Ph.D. programmes.

NAV proactively sought to strengthen collaboration with the University where the research team for the project reported in this paper is based. To do that, they assigned to one of its employees the role of Research Champion, liaising between research and practice and championing research at NAV. To realise this collaboration, the Research Champion got a designated workstation at the University and took part in day-to-day University activities. The physical proximity increased both the formal and informal interactions creating more opportunities for collaboration.

The project reported in this paper was initiated by an invitation from NAV which was the first step in a two-step process. NAV invited the research community to work on user studies for the development and improvement of NAV´s interactions with citizens. For this first step, researchers were asked to submit a short project sketch. NAV shortlisted the most promising sketches and asked research teams to collaborate with NAV employees to develop full project proposals ensuring relevance to the organisation compensating the research time used for collaborative proposal development. Subsequently, NAV selected and fully funded the most relevant projects. The researchers started discussions with the Research Champion during their effort to sketch ideas responding to NAV´s invitation. The Research Champion was not involved in the preparation of that invitation or in the process for selecting projects. The Research Champion shared information about processes, ongoing initiatives, and strategic priorities of the organisation. This led to defining the use of NAV´s chatbot as the focus area. The Research Champion remained involved after the project started, facilitated data collection, participated in workshops, and had a series of meetings with the research team discussing the emerging understanding.

5. Findings

5.1. Action possibilities discovery

The chatbot Frida was introduced by NAV for automation purposes. However, early in the project, it was revealed that some service agents re-appropriated Frida to get support while serving citizens in live chats (e.g., to retrieve or verify information). In other words, Frida was also used in practice for human task augmentation. Service agents and chatbots working at the same time on the same inquiry and not only sequentially (i.e., chatbots handing over chats to humans) allows covering more efficiently a wide range of citizen inquiries. So, what was really the innovation opportunity, was not only the streamlining of handovers, but some deeper conceptual shift in the chatbot’s role in service delivery.

Up till this project, NAV was focused on chatbot interactions with citizens. The organisation has been improving chatbot functionalities through analysing verbal and non-verbal (i.e., clickstreams) chat data collected on a continuous basis, while also engaging in pilot user studies. However, shifting attention to the chatbot as a support for service agents was new. The involvement of researchers was informed by academic thinking and methodology to sensitise practitioners for the discovery of action possibilities. The collaborative process created an understanding which went beyond what the organisation would have been able to achieve on its own. The researchers moved seamlessly into a clinical role although the project started more as “Contract research and expert consulting”, that is a regular project with low involvement on the organisation side and intensive work by the researchers to analyse NAV data. The researchers realised that service agents can be sensitised to discover action possibilities (i.e., affordances) offered by chatbots to innovate from within. They worked in this direction by organising workshops during which service agents shared their own experiences on chatbot use, got sensegiving support by the chatbot technical team and learned about chatbot action possibilities reported in the literature. The researchers facilitated the action possibility discovery process, supported the agents in concretising and articulating the affordances that they identified and shared the results with the organisation. The analysis revealed six chatbot affordances for service agents. These were further classified to affordances related to task “automating” and affordances related to task “augmenting” (i.e., working together with the chatbot for service delivery). The affordances are presented in the following section and summarised in Table 2 together with related chatbot features.

Table 2. Overview of chatbot affordances for service agents identified.

Affordance	Description	Related Chatbot Features
Automating Affordances
Filtering	Possibility to only answer non-trivial questions and questions relevant to own expertise.	Trained to respond to common citizen inquiries. Able to route citizen inquiries to relevant service agents.
Informing	Possibility to retrieve easily up to date information.	Searching in organisation´s knowledge base. Accessing up to date data.
Monitoring	Possibility to identify disruptions in service delivery in near real-time.	Handling hundreds of inquiries simultaneously.
Augmenting Affordances
Delegating	Possibility to assign to chatbot mundane tasks like key information gathering.	Collecting key information from citizens Providing pointers to relevant information sources
Multitasking	Possibility to handle many inquiries simultaneously.	Presenting multiple different conversations Organising responses to different threads
Distilling	Possibility to summarise prior conversations and reduce repetition that enables quicker follow ups.	Summarising conversations with the citizens.

5.2. Chatbot affordances for service agents

5.2.1. Filtering

The chatbot is the first line of response when it comes to handling citizen requests. The chatbot affords filtering as one of the service agents aptly pointed out:

Frida is a filter first that stops quite a few requests and answers quite a few as well.

Some of the requests are fulfilled by the chatbot, some are partially fulfilled and then routed to the right agent, and some are directly routed to a service agent. Some responses to inquiries only take a few interactions with the chatbot. The vast majority of inquiries can be handled completely by the chatbot. During the past year, only one out of five inquiries had to be transferred to a service agent. One service agent explained:

you do not sit down to answer the same question 10 times in a row, because the chatbot manages to say where you find the form, or where you see the case processing time or where you log in … human beings are left with more complex questions, the routine answers are filtered out very quickly. Employees can to a greater extent use their professional competence.

5.2.2. Informing

The chatbot affords quick information retrieval not only for the citizens but also for the service agents. During the interaction with citizens, service agents are able to understand the context of inquiries fast and based on this understanding they often use the chatbot themselves to retrieve or validate relevant information using the organisation´s knowledge base. One of the service agents pointed to this chatbot affordance:

while talking to a citizen, we often use the chatbot for getting quick answers to questions, which we use to solve their [i.e., citizens] problem.

Retrieving information from the organisation´s knowledge base is useful even for the most experienced agents. They can consult the chatbot for the latest regulations, to make sure that nothing has changed.

Yes, the chatbot will always have the latest regulations and so on, but an agent may not be aware of a change or may make a mistake, so the chatbot is used for this

5.2.3. Monitoring

When the chatbot suddenly receives a high number of inquiries on a specific topic it makes it possible to understand that there is something abnormal going on as explained by one of the service agents of the organisation:

the chatbot catches up quickly, if something has gone wrong, a solution is down, something has failed … when we suddenly see a lot of inquiries to come in about one topic, then we [can] act immediately to deal with it in a smooth and good way

Based on information about a new pattern in inquiries, the team creates and publishes new content to address the issue and improve and further develop services overall:

The chatbot coaches create new answers so the chatbot can handle this [new issue]. We report it to the webpage editorial staff, who can publish about it. We create voice messages for those who call in. And so on.

The chatbot provides new possibilities for situational awareness and early detection of service-related issues. As the chatbot can handle hundreds of inquiries at the same time, it only takes a few minutes to accumulate valuable information that would alternatively only become available through the coordination of multiple service agents.

5.2.4. Delegating

The chatbot facilitates conversation-based exchanges but it sometimes fails to identify the context properly leading to dead-ends. Being able to understand context is a critical aspect of chatbot use that can have a significant impact on how effective and efficient the interactions of the citizens are both with the chatbot but also with the service agents. For example, understanding if the citizen is an employee or an employer, a simple but critical information, changes the type of information that should be offered. One of the service agents explained this:

Frida should understand better in which context you make contact … if you ask Frida a question about sickness benefits, for example, then Frida does not necessarily know if you are an employer or an employee. And that kind of information is important because it may be that the answer is a little different if you are an employer

Humans are better in contextualising the inquiries and identifying the issues through proper follow-up questions:

Often, once connected to a service agent, the citizen will try to explain their case better and offer more details, but it is always up to the agent to ask the right questions in order to fully understand the issue at hand and offer the appropriate answers and solution.

The chatbot provides the possibility to agents to delegate mundane conversational tasks (for instance, related to information collection) while retaining control and directing the overall conversation.

5.2.5. Multitasking

The service agents have developed the ability to handle multiple chats in parallel switching attention from one task to another, which can be further supported by making inquiries to the chatbot. This multitasking possibility contributes to service efficiency. As the chatbot can interact with numerous citizens, service agents also can hold several chats at the same time.

The service agents [can] take three chats at the same time. So, it makes the channel more efficient.

5.2.6. Distilling

When a chat is transferred to a service agent, it is important for the agent to check the information that has already been exchanged in order to pick-up from there as explained by one of the service agents:

What we do is to give first an automatic message: you have been transferred to human, my name is so and so and give me a minute to read the transcript of what you talked about earlier. And then we read through quite fast and check what was asked. What information has been given. Then I usually pick up on what has been exchanged and what seems to be the misunderstanding. What seems to be the reason for the transfer.

By summarising the key points of the conversation that took place and presenting the summary to the service agents the chatbot can significantly facilitate their work, enabling them to answer faster:

it would have been an advantage if Frida could give a quick summary of what the questions were instead of us having to read through the entire chat log.

6. Discussion and implications

By being invited to help the organisation to improve interactions with citizens, the researchers have been privileged with an intimate knowledge of the organisational practices and ongoing initiatives. The initiation of this project from the organisation itself is what makes it different from traditional researcher-initiated projects, such as action research or case studies (Schein, 2008, pp. 268, Figure 18.1). Clinical research helps to uncover real “insights” both on the part of the organisation and the researcher (idem). As the organisational participants become active inquirers, they see new areas of relevant information that may never have occurred to themselves or to the researchers in a traditional case study. To explore new forms of human–AI interactions for handling citizen requests the researchers worked collaboratively with the practitioners to generate insights on the action possibilities offered by chatbots and on the potential of hybrid human/AI service teams.

An overview of the clinical research reported in this paper is provided in Figure 1.

Figure 1. Overview of clinical research on chatbot use by NAV service agents.

The intervention consists of sensitising service agents for the discovery of chatbot affordances, concretising, and articulating the perceived affordances, and making these affordances available for service improvement. The discovery of affordances made service agents recognise the potential of hybrid service teams. Recognising the potential of hybrid teams empowers service agents to refactor communications by fusing their own competencies with chatbots’ capabilities. Furthermore, following a double-loop learning process, three lessons-learned were distilled from the project. The lessons are related to human/AI partnerships, theory-based interventions, and institutionalised collaborative research. They can be useful for researchers who want to engage with practice and organisations that want to evolve their technology use engaging with research. These lessons are the main contributions of this paper and presented in the sections that follow.

6.1. Lessons learned

Human/AI partnerships in public service delivery: Encompass both automation and augmentation

The project helped in showing how chatbots can contribute to the improvement of interactions with citizens not only by automating responses, but also by supporting service agents in their own tasks, ensuring complementarities between automation and augmentation (Raisch & Krakowski, 2021). Chatbots can be used as smart personal assistants of service agents. For instance, they can distil the key points of information exchanges with citizens and present them to the service agents enabling faster responses. Chatbots can facilitate multitasking and can also be used for collecting basic information reducing repetitive tasks for service agents. There is significant potential for further advancement in the direction of hybrid service teams and significant benefits to be reaped by having humans and AI join forces (Raisch & Krakowski, 2021; Wilson & Daugherty, 2018). For instance, during human-based live chats, conversations can be processed by bots and suggested answers to citizens’ questions can be presented. Service agents can directly use these suggestions or edit them before responding.

The service agent and the chatbot can work as a hybrid team, bringing together the ability of the chatbot to quickly process information and the ability of the humans to show empathy and assess context. These synergies between service agents and chatbots can be leveraged for refactoring communications by focusing on the interplay within the dyadic human-AI partnership (Baird & Maruping, 2021). Human-AI teams may exceed the performance of either of the parties individually if tasks are allocated by taking into account their relative strengths and weaknesses (Fügener et al., 2021). The focus on human-AI partnerships allows leveraging the conjoined agency between humans and AI (Murray et al., 2021) by deploying chatbots as augmenting technologies alongside their use for automation. This way, live chat efficiency can be improved while retaining the characteristics of good, humane service performance.

The affordances identified in this study provide an empirically grounded basis for advancing human/AI interactions and partnerships (Raisch & Krakowski, 2021) and provide insights on the potential of AI to reshape work (Faraj et al., 2018; Lacity et al., 2021). There are few studies available on human-chatbot interactions and partnerships. These studies do not include the perspectives of service agents experienced with chatbots in practice, instead, they draw from envisioning discussions with service agents before any use of chatbots (Barnett et al., 2020) or infer affordances for service agents through interviews with developers and managers (Waizenegger et al., 2020). We complement existing works by engaging with service agents to identify affordances pointing both to automating and augmenting potentials for developing human/AI hybrid service models.

Theory-based intervention: Affordances as a Sensitising Concept

The intervention entailed sensitising service agents and helping them perceive action possibilities afforded by chatbots. Prior research on affordances was used as the theoretical basis for this. Research findings related to affordance discovery (Lehrig et al., 2017; Mesgari & Okoli, 2019) were operationalised and used to organise activities for helping service agents discover the potential of chatbot technology themselves. Specifically, through a series of workshops, service agents were exposed to the ways chatbots are used by others (as reported in publications or as described by other agents within the same organisation) and were aided to explore further possibilities through discussions with the chatbot development team that provided “sensegiving” support. Schein has pointed out that data gathering is in itself an intervention but more often than not, researchers are not aware of the possible consequences (Schein, 2008). In the project reported in this paper, the research team framed the research process not as a “benign” data collection activity that precedes the intervention but as an intervention by itself. They exposed the effects that a guided discovery process has on users’ understanding of chatbots’ potential and engaged in dialogue on how this understanding can be used for further developing chatbots’ functionality and service delivery processes. The aim was to build joint knowledge; instead of researchers analysing processes and practices to identify areas for technology-enabled improvements and then explaining them to the organisation, the researchers worked with the practitioners facilitating their discovery process being committed to a joint inquiry.

The theoretical insight that perceiving affordances is necessary for their actualisation helped researchers design the intervention. The discovery of affordances by the service agents themselves is an essential part of our intervention. These affordances stem from the experiences of the service agents and reflect the gradual development of their understanding about action possibilities offered by the chatbot. Service agents, as seasoned practitioners, are well placed to use emerging technologies that afford new actions flexibly recombining them to generate new practices achieving transformative effects (Pentland et al., 2022). Facilitating service agents to discover action possibilities is an essential step for moving towards Human–AI partnerships. Through this project, NAV entered a creative process of affordance discovery bringing together technology knowledge with service delivery expertise. The work performed together by practitioners and researchers contributes to bringing chatbots and humans together in hybrid service teams and provides insights about the facilitation of affordances’ discovery.

Institutionalised collaborative research: Research champions can enable clinical research

NAV has a strong track record in research collaborations and has introduced the Research Champion role to establish connections, monitor activities, and sustain good relations with the researchers´ University. The Research Champion is employed by NAV and physically hosted at the university premises. The potential of champions for successful industry-university collaborations is clear, however there are multiple factors to be considered, including institutional factors, trust, fairness, and awareness (Rybnicek & Königsgruber, 2019). Here, we show how the champion was able to successfully collaborate with the researchers and promote clinical research to solve together a problem set by the organisation. Research Champions can develop relationships of high intensity (Santoro & Chakrabarti, 2002) and mediate initial trust formation mechanisms (Hemmert et al., 2014). The Research Champion [also a co-author] was key in connecting research with practice, mobilising both researchers and practitioners and fostering a genuinely participative process. Typically, a client always takes part in the clinical process, but not necessarily involved in “research” (Schein, 2008). Here, the service agents were co-creators of the new knowledge on enacting human/AI partnerships.

NAV has been promoting a research attitude within the organisation and the employees were proactive and reflective in finding solutions and interested in using research insights for their practices. The experience from this project indicates that the creation of Research Champion roles offers a pathway to increase clinical research studies and institutionalise collaborative research. The relationships between researchers, consultants, subjects, and clients can take different forms based on different levels of involvement (Schein, 2008) and a Research Champion can, in general, contribute to multiple projects beyond clinical inquiry. Our Research Champion was also involved in other projects that follow different research designs including contract research, case study research and action research (initiated by researchers). Overall, the champion acted as a scout seeking potential liaisons, monitoring activities both in the research and practice sides and sustaining good relations.

7. Concluding comments

Human/AI interactions have evolved significantly in the past decade. NAV’s chatbot project demonstrates how a major public service organisation in Norway is actively exploring the potential of AI technologies for enhancing the smartification of its services. Research shows that AI applications are currently the most important IS innovations for the public sector (Benbunan-Fich et al., 2020). However, public sector organisations have a moderate track record of IS innovation results and would benefit from a process that rewards disruption from within (Benbunan-Fich et al., 2020; Opland et al., 2022; Vassilakopoulou & Grisot, 2020). To achieve disruption from within, the involvement of public sector practitioners is key and the experiences from the project reported in this paper provide a pathway for involving practitioners and researchers in joint innovation efforts. Prior research suggests that the realisation of smartness in public sector innovation requires exploring and adopting new work practices rather than simply implementing emerging technologies (Velsberg et al., 2020). This clinical inquiry project shows how practitioners can get sensitised to the potential of new technologies to generate new practices with transformative effects. The researchers used the concept of affordances and prior research related to affordance discovery (Lehrig et al., 2017; Mesgari & Okoli, 2019), as an instrument, for sensitising practitioners to action possibilities offered by chatbots and to the potential of deploying hybrid human/AI service teams.

Drawing from the whole research process, the paper contributes by offering three lessons learned on human/AI partnerships, theory-based interventions, and institutionalised collaborative research, that can be useful for researchers who want to engage with practice and organisations that want to evolve their technology use. These lessons pave the way for future engaged research on human-AI partnerships. Extending our work, researchers can leverage next generation IS theories on agent interactions for theory-based interventions to deliver new insights for human-AI partnerships (Baird & Maruping, 2021; Burton-Jones et al., 2021). Our project adds to the body of literature on action-oriented research with practice where the inquiry is driven by organisational needs (Baskerville et al., 2015; Mathiassen & Nielsen, 2008). It shows how IS researchers can contribute, in practical terms, by operationalising theoretical insights to stimulate organisational members to depart from established practices and innovate from within with the use of digital technologies.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was supported by the Norwegian Labour and Welfare Administration (NAV) under the project Frida@work (20/5690) and the Research Council of Norway under the project AI4Users (311680).

Annex:

Method

The study is based on a clinical approach (Schein, 1987, 2008), where the focus of the researchers’ activities at the organisation is to contribute to its improvement. The engagement of the researchers was initiated by an invitation to the research community issued by the organisation in January 2020 with the overall aim to improve interactions with citizens. Responding to this invitation, a mixed team with researchers from three Universities was formed to investigate specifically chatbot interactions. The initiation of this project from the organisation itself is what makes it different from traditional researcher-initiated projects, such as action research or case studies (Schein, 2008, pp. 268, Figure 18.1). Clinical research “involves the gathering of data in clinical settings that are created by people seeking help” (p. 267) (idem). By contrast, in a researcher-initiated case study we would collect data to understand how the organisation is using the chatbot and then offer them our findings as solutions on how they can expand the use of chatbots, without organisational participants becoming active inquirers. The two researchers in the authors’ team of this paper focused on the perspectives of service agents and the possibilities for Human-AI interactions in chat-based customer service (the other two University research teams worked on the perspectives of citizens on the use of chatbots and multi-channel strategy within NAV). The other two authors are practitioners working for the public service organisation and have been involved throughout the project including the early discussions before it started.

The public organisation that invited the research community to contribute in improving interactions with the citizens has significant prior experience with contract research and expert consulting. This mode of engagement with researchers is quite different to clinical inquiries because it entails relatively low involvement from the organisation´s side. What the organisation wants in this model is data and information and the research team is hired to be an expert in providing it (Schein, 2008). Nevertheless, for this study, the organisation was willing to allocate resources and work together with the researchers to explore the possibilities for better leveraging technology in service delivery. One of the participants in this study (who is also one of the two co-authoring practitioners) has a special role within the organisation, liaising between research and practice as Research Champion. This proved valuable in communications with key organisation employees. The activities were driven by the organisation’s agenda and the problem was narrowed down after a series of discussions between researchers and organisation representatives. After this process, the researchers formulated the aim of the inquiry in a way that is easily communicated with the organisation’s employees and faithful to the overall organisation’s agenda: “How can we better leverage chatbot technologies for Human-AI interactions in chat-based customer service?”.

Workshops where both researchers and employees of the public organisation participated were organised. In addition, during the project, a series of interviews with service agents were conducted to gain the much-needed contextual understanding regarding the use of chatbots by the service agents and a literature review on published research related to action possibilities afforded by chatbots was performed. To facilitate communications, to better capture cultural aspects within the organisational context and to ensure time availability for engaging with the organisation, research assistants that are native speakers were hired for the project. The research assistants also worked on the literature review. During the workshops, the employees reflected on their experiences. They also recalled different adaptations in technology configurations and service arrangements implemented gradually after the chatbot’s introduction. Since the process was initiated by the organisation, the participation in workshops was facilitated by the management and the Research Champion. The research team and the practitioners were fully involved in the process having joint responsibility for advancing the effort. Figure A1 presents the pathway to sensitising the service agents for discovering chatbot affordances.

Figure A1. Pathway to sensitising service agents for discovering chatbot affordances.

In total, seven workshops were organised. The participants from the practice side were from the management level, the IT team, and service agents. Several iterations were required to further define and concretise the problem and the first three workshops were dedicated on this. Activities related to formulating the problem, inquiring into the situation, and refining the problem took place in collaboration with management employees and service agents. In the next workshop, the participants were only service agents to clarify the nature and scope of human-chatbot interactions and capture the knowledge and expertise of the employees that use the chatbot. In the last three workshops, we engaged in interactions through continuous questioning and probing at individual and group levels. During these workshops, we intervened by sensitising the service agents for discovering the action possibilities that emerged when using the chatbot.

The workshop transcripts were analysed, and a consolidated list of action possibilities was developed. The different action possibilities were clustered in overall themes. The possibilities afforded to service agents were further classified to possibilities related to task “automating” and possibilities related to task “augmenting” by working together with the chatbot for service delivery (Figure A2). The affordances identified stem from the experiences of the service agents and reflect the gradual development of their understanding about action possibilities offered by the chatbot. The discussions allowed the whole practitioner/researcher team to better understand and conceptualise human/AI partnerships and to develop abstractions of the learnings from the organisation’s experiences with chatbots. The concept of affordances and prior research related to the affordance discovery and perception were used for sensitising participants around the action possibilities offered by chatbots. The service agents were supported to perceive chatbot affordances and this allowed them to envision novel service configurations that leverage chatbot technology. In the final round of data analysis, we distilled three key lessons learned that can be useful for organisations that want to evolve their technology use, stimulate innovation, and engage in collaborative research with academia and for researchers that want to engage with practice.

This study has received approval from NSD – Norwegian Centre for Research Data (603215).

Figure A2. Data analysis process and results.