The impact of AI-based conversational agent on the firms’ operational performance: Empirical evidence from a call center

ABSTRACT

Artificial intelligence (AI) based conversational agent is widely used in the service industry. Although some studies have investigated the impact of AI on customers, little research has documented the concrete effects of AI-based conversational agents on firms’ operational performance. To fill this gap, the main goal of this paper is to investigate the impacts of AI-based conversational agent on firms’ operational performance. In addition, the moderating effect of time blocks is also investigated. We address questions using the event study method and a proprietary data set from a telecom firm in China. Results show that the introduction of an AI-based conversational agent increases the average call length and has no significant influence on call numbers. Moreover, there is a heterogeneous effect among different time blocks. Our findings provide important implications for the operational management of call center.

Introduction

Artificial intelligence (AI) has become a hot topic in the service industry. The AI-based applications are widely used in the service field (Huang and Rust 2018; Wirtz and Zeithaml 2018). For example, the AI-based conversational agent, an AI application can simulate human conversations through voice commands, has been widely introduced into the service line (Leviathan and Matias 2018). With the development and maturity of Natural Language Processing (NLP) and machine learning (ML). AI-based conversational agent is becoming more and more popular, and affects every aspect of our daily lives such as finance, medical, transport, and service field (Hutson 2019; Maedche et al. 2019; Makridakis 2017; Taddeo and Floridi 2018). AI-based conversational agent has a series of advantages, which contains huge potential business value (Canhoto and Clear 2020). Research shows that AI-based conversational agent can dramatically reduce current global business costs (Techlabs 2017). AI-based conversational agent has a number of advantages over human. For instance, AI-based conversational agent can provide 24 hour service, and is free from interference from external environment (Meuter et al. 2005). Because of these advantages, more and more service enterprises begin to introduce AI-based conversational agent.

However, the performance of AI-based conversational agent in practical application scenarios has not received enough attention, especially in firm operational performance. The impacts of AI-based conversational agent on firm operational performance is unclear. On the one hand, AI-based conversational agent has notable advantages including stability, efficiency, and accuracy. Therefore, AI-based conversational agent can improve a firm’s user experience, such as simplifying the process of interaction and shorter reacting time. On the other hand, Algorithm discrimination is an important problem in the based conversational agent promotion (Dietvorst, Simmons, and Massey 2018; Kawaguchi 2020). That is, humans may prejudice that AI-based conversational agent lack personal feeling and empathy, perceiving bots as less trustworthy with payment information and product recommendations (Luo et al. 2019). Customers are unwilling to talk with computer programs for personal needs (Longoni, Bonezzi, and Morewedge 2019) or let chatbots assist in purchase decisions (Luo et al. 2019), which are not conducive to the marketization process of AI-based conversational agent. Although a wealth of information system (IS) research has investigated the impact of AI applications on customer behaviors, such as purchase behavior (Luo et al. 2019) and search behavior (Sun et al. 2019), empirical evidence documenting the concrete effect of AI-based conversational agent on operational performance is largely lacking.

As more and more enterprises introduce AI-based conversational agent in operation management (Apell and Eriksson 2021; Brynjolfsson, Hui, and Liu 2019; Dubey et al. 2020; McLean and Osei-Frimpong 2019; Rust and Huang 2012). It is very important to explore the influence of artificial intelligence on enterprise operation and management. Therefore, The main purpose of this study is to explore the impact of AI-based conversational agent on enterprises’ operational performance. Specifically, we mainly explore the following questions.

RQ1:

Whether AI-based conversational agent increases or decreases call center operational performance?

RQ2:

Whether the influence of AI-based conversational agent on call center operational performance is moderated by other factors (i.e., time blocks)?

The purpose of this study is to answer these questions. Inspired by relevant research such as Devaraj, Fan, and Kohli (2006), Khudyakov, Feigin, and Mandelbaum (2010), and Aksin, Armony, and Mehrotra (2007), we use daily incoming calls, average call length respectively to measure the company’s operational performance. We work with a Chinese telecommunications company that has more than 10 million users. This company officially introduced AI-based conversational agent on January 1, 2019. We get a lot of operational data from call centers. We used event method to explore the impact of the introduction of AI-based conversational agent on the operational performance of call centers. We got three main findings. First, The introduction of AI-based conversational agent increases the average call length. That means the number of people served per unit of time is down. Call center may face the risk of insufficient supply of service capacity. Second, we find that the AI-based conversational agent has no significant influence on average daily incoming calls. This means that AI-based conversational agent does not increase the need for human customer service. Third, the influence of AI-based conversational agent on operational performance is moderated by the time blocks. The rest of this article is shown below.

First, recapitulate the literature streams on AI-based conversational agent and call centers’ operational performance. Subsequently, the research background, data source, structure and research method are described in detail. Finally, the results of estimation of AI-based conversational agent on call center operational performance are reported. Moreover, the implications for theory and practice are also discussed.

Literature Review

There are three streams of research in the literature that are relevant to our study. The first stream of research focuses on the impact of AI-based conversational agent on customer responses. AI-based conversational agent has a very outstanding interactive ability, it can imitate human chat with users (Araujo 2018). As a front-line server, it can respond quickly to users’ questions via text or voice. The report says smart conversational agent can save nearly 30% of customer service costs (Techlabs 2017). With the continuous improvement of artificial intelligence technology, the function of AI-based conversational agent is becoming more and more powerful. Most existing AI-based conversational agents are already perfectly capable of doing the job of human. For instance, AI-based applications can help enterprises promote their products (Luo et al. 2019) and provide investment advice to an investor (Ge et al. 2021).

AI-based conversational agent has attracted wide attention not only from the industry, but also from the academic community. The influence of AI-based conversational agent on user behavior has become the focus of research. Like most IT technologies, AI-based conversational agent has its pros and cons. On the one hand, AI-based chatbot can help enterprises improve users’ purchase behavior and search behavior (Sun et al. 2019). The combination of AI-based coach and human l coach can improve the training effect (Luo et al. 2021). On the other hand, AI-based conversational agent inevitably has some downsides. For example, the disclosure of AI-based conversational agent will have a negative impact on users’ purchase behavior (Luo et al. 2019). The anthropomorphism of AI-based conversational agent may also harm customers’ satisfaction (Crolic et al. 2022). It not only has a negative impact on users, but also has a negative impact on employees. For instance, the deployment of AI-based conversational agent would reduce employee’s performance (Tong et al. 2021). The emotional intelligence has a significant effect on employee retention and performance (Prentice, Dominique Lopes, and Wang 2020). The implementation of AI would decrease employees’ perceived fairness and satisfaction (Köchling, Wehner, and Ruhle 2021). However, few studies have explored the impact of AI-based conversational agent on enterprise operation performance from the perspective of enterprise operation management.

The second stream of research focuses on the operational performance. Operational performance is the core content of enterprise operation management. How to improve operational performance is the main concern of managers in service industry and manufacturing industry. Operational performance is not only concerned by enterprises, but also by the academic community. New technology is often seen as an important driver of business growth (Baffour et al. 2020). Therefore, the relationship between the implementation of new technology and firm operational performance has attracted the attention of many scholars. For example, Belekoukias, Garza-Reyes, and Kumar (2014) investigated the impact of lean methods and tools on the operational performance of manufacturing organizations. Cottelleer (2006) investigated the impact of enterprise systems on the operational performance. The impact of IT on operational performance has always been an important issue (McAfee 2002). McAfee (2002) found that the adoption of information technology will have a significant impact on business performance. In addition, the implementation of ERP will improve enterprise performance (Cotteleer and Bendoly 2006; Cottelleer 2006). Liang, You, and Liu (2010) found that information technology (IT) has a significant effect on firm performance. However, the effect of AI-based conversational agent implementation on business performance has not been fully studied, especially in the context of call center.

The third stream of research focuses on the call center. Call center is an important channel for enterprises to provide customer service (Tezcan and Behzad 2012). The call center receives a large number of calls from customers every day that they have to hire a large number of front-line operators to improve their service capacity. This means that call centers spend a lot of money on staff. It was said that 60–80% of the cost of a call center is taken up by employee salaries (Aksin, Armony, and Mehrotra 2007). The rapid increase in their number of users and the explosive growth of user data has placed call centers of various firms under an enormous amount of pressure in terms of supplying services effectively. There is no doubt that the huge manpower cost is not good for the long-term development of call center. The operational performance of call center is one of the uppermost priority (Akşin et al. 2017). In order to improve operational performance and reduce costs, enterprises have taken a series of measures such as outsourcing and process optimization. The impact of these measures on enterprise operation performance also has attracted scholars’ attention. For example, A method for staffing and routing based on linear programming (LP) to reduce the total cost was proposed by Bassamboo, Harrison, and Zeevi (2006). Whitt (2006) also proposed simple methods for staffing a single-class call center with an uncertain arrival rate and uncertain staffing due to employee absenteeism. However, the relationships between call center operational performance and AI-based conversational agent have been largely overlooked by researchers to date.

However, the impact of AI-based conversational agent on call centers, especially on operational performance, is still unclear. With the intensification of market competition, AI-based conversational agent plays an increasingly important role. Enterprise managers hope to realize the goal of advancing the core competitive ability with the introduction of AI-based conversational agent. Therefore, it is important to examine how the introduction of AI-based conversational agent affects firms’ call center operational performance.

Empirical Methodology and Analysis

Background

In recent years, more and more enterprise call centers have begun to introduce AI-based conversational agent to front-line service posts. Our cooperative enterprise is a large telecom operator in China, which currently has more than 10 million customers. The company’s call center receives a large number of users every day. Therefore, this company employs a large number of operators to provide services, which means that the enterprise has invested a large amount of money in human costs. In this context, enterprise managers hope to reduce costs and improve efficiency by introducing AI-based conversational agents. The AI-based conversational agent has an optimized female voice designed for an appealing pitch, tone, speed, and intonation to engage customers. It allows customers to ask questions or express demands by voice directly. Customers can consult and give feedback regarding changes to their accounts, checking funds, reporting malfunctions, etc. Upon receiving a customer request, the AI-based conversational agent responds. If customers are dissatisfied with the AI responses, they can request to transfer to a human.

Before the introduction of AI-based conversational agent, the customer service structure of the call center was a combination of interactive voice response (IVR) and human. IVR prompt the customers to press the key by voice, judge the user’s intention according to the key system, then go to the knowledge base to match the answer and return it to the user in the form of voice. One of the main shortcomings is that customers need to press the button continuously to acquire the required knowledge level by level. The company carried out a random experiment on December 19, 2018 to test the actual performance of AI-based conversational agent. Specifically, customers with 1 and 7 were selected to use the AI-based conversational agent while those with 3, 5, or 9 were selected to use the IVR system on December 19, 2018. Before that day, all service calls were first connected to the IVR system where customers followed voice instructions and used keypad presses to select their service. In the period between December 19, 2018, and January 9, 2019, the call center introduced an AI-based conversational agent to replace the IVR system for a portion of customers. After January 15, 2019, the IVR system was completely replaced by the AI system. In order to facilitate understanding, we have made a schematic diagram of the human-computer interaction process (please see Figure 1).

Figure 1. The service flow chart of call center.

Data Collection and Processing

We have obtained a large amount of user data from the call center, which includes user demographic information, such as age, gender and open year. In this paper, customers are divided into two groups by the last phone digit, specifically, “Group 1” includes the users whose last phone number digit is 1 or 7, and “Group 2” includes users whose last phone number digit is 3, 5, or 9. We have a total of 69,096 users, including 28,288 in the group 1 and 40,808 in the group 2. We ran a randomization check on the users of Group 1 and Group 2. Table 1 shows the mean test results of the two groups.

Table 1. Descriptive statistics.

Variables	Data type	Explanations	Min	Max	Mean	Median
Age	Integer	Actual age calculated based on ID card information	16	70	40.02	37.9
Gender	Binary	l = male, 0 = female	0	1	0.37	0
Open year	Integer	Length of time from registration to now	0	21.67	6.83	6.2

Table 2 shows that there is no difference in age, gender, and open year between Group 1 and Group 2. The T-test results indicate that the sample allocation is sufficiently random.

Table 2. Randomization check.

Group	Age	Gender	Open year
Group 1 (last_digit=1,7)	39.89	0.36	6.74
Group 2 (last_digit=3,5,9)	39.97	0.36	6.77
T-value	0.99	−0.19	1.10

In this paper, we have built two variables to measure the operational performance of the call center: one is the Daily incoming calls_t, the other one is the Average call length_t. The Daily incoming calls_t refers to daily number of calls at the human on day t. The average call length_t is calculated by the equation 1 (please see Table 3). It refers to the average service duration between customers and humans in a call center. These two indicators play an important role in guiding the allocation of human resources and the scheduling of employees. For example, if the average call duration is too long, the enterprise should adopt appropriate incentive measures to increase the enthusiasm of human. If the incoming daily calls changes significantly, the company should adjust the scheduling plan in time. Table 4 shows the description of the variables.

Table 3. The description of variables.

Variables	Description
Average daily call lengtht,g (ALt,g)	Average daily call length of group 1 or group 2 on day t
Average call length differencet (ALDt)	The difference of average daily call length between group 1 and group 2 on day t
Abnormal daily incoming callst (ADCt)	The number of abnormal daily incoming calls of group 1 on day t
Abnormal average call length difference of each dayt (AALDt)	The difference of average abnormal daily incoming calls between group 1 and group 2 on day t
Predicted daily incoming callst (PDCt)	The predicting daily incoming of group 1 on day t
The mean of abnormal daily call lengtht (M1)	The abnormal daily call length of group 1 on the time period of post-event
The mean of abnormal average call lengtht (M2)	The average abnormal call length of group 1 on the time period of post-event

Table 4. A table of function sequences.

Equations	Description
Equation 1: $A{L}_{t,g}=\frac{\sum _{i=1}^N{L}_{i,t,g}}{DI{C}_{t,g}}$	This equation is used to calculate the average daily call length. i represents the i incoming call, t represents the t day, and g represents the group of the caller.
Equation 2:$AL{D}_t=A{L}_{t,1}-A{L}_{t,2}$	This equation is used to calculate the average call length difference. t represents the t day.
Equation 3: $AD{C}_t=PD{C}_{t,1}-PD{C}_{t,2}$	This equation is used to calculate the abnormal daily incoming calls. t represents the t day.
Equation 4:$AAL{D}_t=PA{L}_{t,1}-OA{L}_{t,2}$	This equation is used to calculate the abnormal average call length difference of each day. t represents the t day.
Equation 5: $OD{C}_t=\alpha +\beta X_t+{\epsilon }_t$	This equation is used to calculate the $\stackrel{⌢}{\beta }$. t represents the t day.
Equation 6: $PD{C}_t=\alpha +\hat{\beta }{X}_t+{\epsilon }_t$	This equation is used to calculate the predicted daily incoming calls. t represents the t day.
Equation 7:$M_1=\sum _{t=1}^{\hspace{0.17em}}\frac{AD{C}_t}{N_1}$	This equation is used to calculate the mean of the abnormal average daily incoming calls. t represents the t day. N represents the total number of days.
Equation 8: $M_2=\sum _{t=1}^{\hspace{0.17em}}\frac{AAL{D}_t}{N_1}$	This equation is used to calculate the mean of the abnormal average call length difference of total days. t represents the t day. N represents the total number of days.

Dependent Variables

There are two important dependent variables in this analysis: Daily incoming calls_t,1, which refers to the total number of calls from the Group 1 per day and Average call length_t,1, which refers to the average temporal length of every call from beginning to end.

Explanation Variables

The variable Daily incoming calls_t,2 refers to the total call numbers that come from Group 2 per day and Average call length_t,2 refers to the average length of every call, similar to Group 1.

Model Development: Event Study

Event study method is a classical statistical method. It was widely used to see how the stock market reacts to the announcement of block share transfers (MacKinlay 1997). This method is mainly used to compare the difference between observed value and predicted value. In recent years, this method has been widely used in information system research (i.e., Im, Dow, and Grover 2001; Lehmann and Schwerdtfeger 2016; Nicolae et al. 2017). This paper also used this method to study the impact of the event of AI-based conversational agent implementation on operational performance at the call center. According to the time node introduced by AI-based conversational agent, the time period is divided into two parts, which are before and after the event (please see Figure 2). In this study, the event took place on 19 December 2018. Therefore, the pre-event period is from 1 November 2018 to 18 December 2018 and the post-event is from 19 December 2018 to 9 January 2019.

Figure 2. Timeline of event study.

The data of period of pre-evnet is mainly used to predict the results of the period of post-event. Specifically, we used the amount of daily incoming calls and average call length in the group 1 and the group 2 received in the pre-event (the time period before the event node) to predict the amount of daily incoming calls and average call length they received in the post-event (the time period after the event node). We then compared the difference between the predicted and observed values. The difference between the predicted and observed value was defined as abnormal daily incoming calls and abnormal average call length. The two variables are calculated as follows.

First, we need to know the relationship between the number of calls in the group 1 and the number of calls in the group 2. Therefore, we use the equation 5 (please see Table 4) to obtain the correlation of daily incoming calls between the group 1 and group 2. Where X represents the observed daily incoming calls from group 2. ODC_t represents the observed daily incoming calls from group 1. $\alpha$ is a constant term. $\epsilon$ is the residual term. We estimated the coefficient by OLS regression.

Next, we use the coefficient $\hat{\beta }$ to calculate the predicted daily incoming calls of the Group 1 in the period of post-event. Specifically, we plug the coefficient $\hat{\beta }$ into the equation 6, we get the predicted daily incoming calls of Group 1. Where X represents the observed daily incoming calls from group 2. PDCt represents the observed daily incoming calls from group 1. $\alpha$ is a constant term. $\epsilon$ is the residual term. We also estimated the coefficient by OLS regression.

According to equation 5 and 6, we get the predicted daily incoming calls of group 1 after the occurrence of the event. We then use the observed daily incoming calls and predicted daily incoming calls after the event date to calculate abnormal daily incoming calls of group 1. The measure of abnormal daily incoming calls of group 1 is shown in equation 7 and 8. The “abnormal” post-event daily incoming calls were computed as the difference between the actual observed daily incoming calls after the event and the “normal” expected daily incoming calls. The abnormal daily incoming calls is calculated by equation 3. Average call length is another important variable for measuring operational performance. The abnormal average call length difference of each day is calculated by equation 4. If the introduction of an AI-based agent has no significant effects on the company’s operational performance, both the average abnormal daily incoming calls and the abnormal average call length difference should obey the students’ distribution with a zero mean. Therefore, we use the equation 7 and equation 8 to test whether M1 and M2 conform to the Gaussian distribution with a zero mean.

There are four key steps in this study. Step 1. We divided all users into two groups based on the end of their phone numbers. The group 1 consisted of users whose phone numbers ended in 1,7. The group 2 consisted of users whose phone numbers ended in 3,5,9; Step 2. Equation 1 was used to calculate the average call length. The number of daily calls between the two groups is directly collected. Step 3. Equation 5 and 6 were used to calculate the predicted daily calls of group 1. Step 4. Equation 7 and 8 were used to calculate the mean of the average abnormal daily incoming calls and the mean of the average call length difference of total days. Then, the normal distribution test is carried out on M1 and M2 respectively to determine whether the significant value is 0.

Results

The first column of Table 5 provides the mean abnormal call number value post-implementation of the AI-based conversational agent together with the corresponding T values. According to equation 8, the abnormal call length is the predicted average call length minus the observed average call length. Table 5 shows that the introduction of AI-based conversational agent reduces the abnormal call length (−7.23**, P < .05). This indicates that the observed average call length is higher than the predicted average call length, which implies that the AI-based conversational agent increased the call time. The AI-based conversational agent may manage more simple tasks while complex tasks are left to the humans. Table 5 also shows that AI-based conversational agent implementation increases the number of calls per day, but not significantly (P > .05).

Table 5. Results.

	Mean	T value
M1	−19.63	−0.56
M2	−7.23**	−4.53

Heterogeneity

We next tested for heterogeneity among time blocks. The hours between 9:00 P.M. and 8:00 A.M. the following day are not staffed by humans, so we divided the remainder of the day (business hours) into 13-time blocks (Table 6). We used the event study method to analyze the abnormal daily incoming calls and the abnormal call length difference in each time block.

Table 6. Abnormal call length and daily incoming calls by time blocks.

Time block	Mean of abnormal daily incoming calls	Mean of abnormal call length difference
8:00 A.M.-8:59 A.M	−2.12(−0.54）	−7.81 (−0.96)
9:00 A.M.-9:59 A.M	8.05 (1.70)	−5.62 (−1.63)
10:00 A.M.-10:59 A.M	1.10 (0.19)	−0.14 (−0.02)
11:00 A.M.-11:59 A.M	3.54 (0.46)	1.88 (0.38)
12:00 P.M.-12:59 P.M	4.40 (0.71)	−16.84**(- 3.039)
1:00 P.M.-1:59 P.M	2.06 (0.46)	−3.51 (−0.76)
2:00 P.M.-2:59 P.M	5.85 (1.74)	−3.82 (−0.34)
3:00 P.M.-3:59 P.M	12.40**(2.12)	1.50 (0.26)
4:00 P.M.-4:59 P.M	5.37 (1.07)	−15.89**(- 2.33)
5:00 P.M.-5:59 P.M	2.65(0.43)	−16.76**(- 2.01)
6:00 P.M.-6:59 P.M	−6.64**(−2.61)	−4.73 (−0.74)
7:00 P.M.-7:59 P.M	−3.80 (−0.69)	−1.24 (−0.08)
8:00 P.M.-8:59 P.M	2.21 (1.30)	−26.05**(- 2.52)

The moderation effect we observed is shown in Table 6. Column (2) shows where a significant moderation effect emerges from 6:00 P.M. to 6:59 P.M. Specifically, the introduction of AI-based conversational agent increases the incoming calls between 6:00 P.M. to 6:59 P.M. Moreover, the introduction of AI-based conversational agent decrease the incoming calls between 3:00 P.M., to 3:59 P.M. Column (3) shows that the introduction of AI-based conversational agent increase the average call length in the periods of 12:00-12:59 P.M., 4:00- 4:59 P.M., 5:00-5:59 P.M., and 8:00-8:59 P.M. The results of heterogeneity decomposition show that the introduction of AI-based conversational agent has different influences on different time blocks.

Robustness Check

We conducted a “placebo” test based on an arbitrary AI-based conversational agent implementation date and tested whether this altered the daily call numbers and average call length in the pseudo-after period. We tested the period from November 1, 2018, to December 18, 2018, as the pre-period with December 12, 2018, as the pseudo-AI implementation date. Using the operational performance data before this date, we re-estimated the abnormal operational performance in the new post-period after December 12, 2018, using our original model.

Table 7 shows the falsification test results. The placebo did not produce any significant abnormal changes in operational performance in the pseudo-post period.

Table 7. Falsification test.

	Mean	T value
abnormal daily incoming calls	−12.46	−0.88
abnormal call length difference	−.91	−0.57

***, ** and * represent significance at the 1%, 5%, and 10%level, respectively

Conclusion

With the development of artificial intelligence technology, more and more AI-based conversational agent is introduced into the call center. The operational performance of the call center is very important to the enterprise. However, few studies have focused on the impact of AI-based conversational agent on the operational performance of call centers. To fill this research gap, this paper explores the impact of the introduction of AI-based conversational agent on the operation performance of call center. We measure the operational performance of call centers using two key indicators (average call length and daily incoming calls) that affect the allocation of human resources. Based on objective data from enterprises and event study method, we conducted an empirical study on the impact of the introduction of AI-based conversational agent on enterprise operation performance. The empirical results of this paper show that the introduction of AI-based conversational agent will affect the average call length, but have no significant impact on the daily incoming calls. Specifically, the introduction of AI-based conversational agent would increase the average call length. The most likely explanation is that the user spent a certain amount of time describing the problem. Moreover, the introduction of AI-based conversational agent has no significant influence on daily incoming calls. The most likely explanation is that customer service requests are not affected by the current AI-based conversational agent in the call center. In addition, we found that the impact of AI-based conversational agent on average call length is heterogeneous in different time blocks.

Theoretical and Practical Implications

Our research has important theoretical and practical implications. This research has the following theoretical implications. First, we enrich the research literature on new technologies and business operational performance. New technology and business performance have always been the focus of attention in information system field. Existing studies show that the implementation of new technologies will have a corresponding impact on the operational performance of enterprises. For example, the ERP would improve enterprise performance (Cotteleer and Bendoly 2006; Cottelleer 2006). However, the research on artificial intelligence and enterprise operation performance is largely lacking. Our study is one of the few literature to explore the relationship between the implementation of artificial intelligence technology and business operations.

Second, we extend the research literature on call center operations management. Call center operations management has an important impact on the long-term development of enterprises (Akşin et al. 2017). Scholars have carried out a series of researches on how to improve the operation and management of call center. For example, Bassamboo, Harrison, and Zeevi (2006) proposed a method for staffing and routing based on linear programming (LP) to reduce the total cost. Whitt (2006) also proposed simple methods for staffing a single-class call center with an uncertain arrival rate and uncertain staffing due to employee absenteeism. However, few scholars have studied operational performance improvement from the perspective of technology introduction. Since the AI-based conversational agent was introduced into the call center in recent years, there are relatively few researches on the AI-based conversational agent of the call center, especially the researches on the impact of AI-based conversational agent on the operation management of the call center. Therefore, our work is among the few papers which focus on the impact of AI in a call center.

Third, we extend the literature on the impact of AI technologies. In recent years, artificial intelligence technology has been applied to various fields. A large amount of research on artificial intelligence has focused on the effects of AI technology on user behavior (Luo et al. 2019; Sun et al. 2019). However, few studies have looked at the impact of AI technology on the enterprise level. To our best knowledge, this is one of the very few studies that investigate the impact of AI-based conversational agent on the enterprise level.

Our research also has important practical implications. First, our research has found that the introduction of AI-based conversational agent in call centers will increase average call length. This result means that humans spend more time on each call. The increase of average call length is not good news for human because their pay structure is based on the total number of calls rather than the total number of hours served. Therefore, in order to prevent humans from being passive, managers should take corresponding incentive measures such as raise employees’ salaries appropriately. Second, the introduction of AI-based conversational agent in call centers has different influence on the daily incoming calls in different time blocks. In some time blocks (i.e., 6:00 P.M.-6:59 P.M), the number of calls increased due to the introduction of AI-based conversational agent. This result means that the call center may face insufficient service capacity during these time blocks and user’s wait time will be longer. Therefore, managers should adjust the existing employee schedule according to the incoming calls in each time blocks. On the one hand, it can help enterprises optimize the allocation of human resources, and on the other hand, it can help enterprises reduce the waiting time of users and improve the service experience of users.

Limitations and Future Research

Our study inevitably has some research limitations. First, the data in this paper are from the call center of a telecom enterprise, so the universality of our results is easily questioned. The impact of AI applications could be influenced by environments (Yadav and Pavlou 2020). Therefore, whether the results are applicable to the call centers of other types of enterprises (i.e., bank and airlines) remains to be tested in the near future. Second, the operation performance of call center includes many indicators. Due to the limitation of data acquisition conditions, it is difficult for us to obtain indicators other than the existing two variables (daily incoming calls and average call length). Therefore, this paper only explores the impact of the introduction of AI-based conversational agent on daily incoming calls and average call length. Therefore, if conditions permit, scholars can conduct more in-depth research on other indicators (i.e., the satisfaction levels of the users) of call center operation performance.

Authorship Contribution Statement

Zhenyuan Zhang: Conceptualization, Formal analysis, Writing-original draft, Methodology, Software.

Bin Li: Formal analysis, Validation, Visualization, Methodology

Luning Liu: Funding acquisition, Resources, Supervision, Writing-review & editing, Project administration.

Acknowledgement

This research was supported by the grants from the National Natural Science Foundation of China (#72034001, #71974044), Heilongjiang Provincial Natural Science Foundation of China (#YQ2020G004) and the Fundamental Research Funds for the Central Universities (Grant No. HIT.HSS.DZ201905).

Disclosure Statement

No potential conflict of interest was reported by the author(s).