Event Assignment Based on KBQA for Government Service Hotlines

ABSTRACT

Government service hotlines are closely related to people’s lives, and the hotline events assignment task is very important in China, which affects people’s satisfaction with the Chinese government. To address the challenge of improving hotline event assignment accuracy, which is currently hindered by the lack of prior knowledge in the traditional direct departmental assignment approach, we introduce a novel hotline event assignment model based on Knowledge Base Question Answering (KBQA) to leverage prior knowledge and enhance assignment performance. The model extracts event key information by using event extraction module, and the key information of event extracte by this module is used to construct historical events knowledge graph. Then we employ subgraph retrieval module and text retrieval module to obtain relevant prior knowledge from the knowledge graph and “sanding” text respectively. The integrated prior knowledge is then used to predict department scores, guiding the selection of the optimal assignment department through ranking. The experiment results show that the proposed method outperforms existing baseline methods. Meanwhile, the Knowledge Graph (KG) edge elimination experiment also shows that the proposed model is better than other baseline models in terms of incomplete KG.

Introduction

Government service hotline is directly oriented to businesses and the public, and serves as an important channel to report problems, make suggestions, facilitate government decision-making and help solve public service issues. It has played a positive role in addressing pressing concerns for enterprises and people in recent years (Meng, Huang, and Zhang 2021). In the past, citizens in China had to dial 12,369 when encountering noise pollution, 12,315 when encountering consumer rights protection, and 12,319 when encountering occupational behaviors. In the past, when reporting problems to relevant departments, the general public need to make different calls by category. In order to simplify the government service hotline system to improve service efficiency and quality, Chinese government has canceled or merged several existing service hotlines at the end of 2021. Nowadays, the public only need to dial only one phone number to perform operations such as consultation, complaints, reflections, and suggestions. This phone number is called “12345,” which is a 24-hour government service hotline. The hotline is an important window for the municipal government in China to connect with the people, and aims to promote the openness of government affairs, to improve administrative efficiency (Ma, Li, and Shen 2020).

Chinese government service hotline adopts the manual assignment mode. When receiving a call from a citizen, a hotline operator need to keep textual record of event information contained in a hotline such as event time, event address, event content, and event classification, and then assign each event to a corresponding government department (i.e., determine a department to deal with the event) immediately. Because of the shortage of the service seats compared with the developed countries, the quality of Chinese government service hotline assignment still needs to be improved. For example, at present there is only about 50 operators who work for the Wuhu (a city in China) government hotline day and night, and they deal with 600,000 hotline events every month averagely. After merging several existing service hotlines, it puts forward higher requirements for a hotline operator. The accuracy of manual event assignment depends not only on his exact judgment on the classification of citizens’ demands but also on an in-depth understanding of the functions of various government departments. The textual contents recorded by a hotline operator for different hotline events vary in length; most of them have problems such as unclear description and incomplete elements. Thus, each hotline operator requires strong business skills and personal comprehensive quality, needing not only proficiency in communication and summarization skills, but also familiarity with the division of responsibilities of government departments. Unfortunately, due to the imperfect mechanism of operator training, it is difficult for a hotline operator to select the correct responsible department from dozens of departments to complete event assignment on the basis of fully understanding of citizens’ demands. The second hotline event assignment will occur when it is assigned to a wrong department. It will cause disputes between departments as well as reduce public satisfaction with the government. With more and more calls accepted by Chinese government hotline, it is significant to develop an automatic event assignment approach that can accurately determine the corresponding responsible department with the purpose of helping the hotline operators to improve service quality.

With respect to most government hotline event texts, the sentences in an event text are often described around the same event subject. However, the text often contains multiple types of key elements, and such elements are often discrete and distributed in different locations of the description text. In order to better integrate these types of key information, language models such as BERT-like model are used to discover important semantic information in the event text. Directly encoding the event text often results in the model not being able to capture these key information effectively. Even if the key information in the event text can be captured, using a classification-based model to directly establish associations between events and departments may still fail to perform well, because the semantic information contained in the department name itself is relatively limited, and information about the responsibilities of some departments is not available from their names. Thus, the model cannot effectively find the potential association. Further, after considering the above factors together to build an event assignment model, there is a possibility that by focusing only on the current event, more effective potential information cannot be obtained from the associated events, thus affecting the accuracy of the model.

As a new form of knowledge storage and representation, knowledge graph (KG) has been effectively applied in many actual industrial usage scenarios, and is one of the key technologies for the next wave of technology development (Wang et al. 2017). Thus we introduce knowledge graph as a way of encoding and storing historical prior knowledge, and propose a KBQA-based event assignment model for Chinese government hotline (Zhou et al. 2017). As a structure that can effectively store and represent external knowledge, knowledge graph can provide more valuable decision support for constructing relationships between events and departments. Automatic and accurate assignment of the events can not only reduce operating costs of the government hotlines, but also optimize the process and resource allocation of government hotlines. This will allow the government hotlines to be connected faster, assigned more accurately, and handled more effectively, and enhance public recognition of the government. Our model achieved optimal results on two hotline assignment datasets, and on the Dataset1 Hits@1 index achieves 88.66%, F1 index achieves 91.32%, on the Dataset2 Hits@1 index achieves 85.52%, and the F1 index achieves 88.77%. Meanwhile, knowledge graphs’ incomplete experimental results show that our model also has a high effect on incomplete knowledge graphs. The main contributions of this paper include the following:

1. Constructing an event extraction module to extract key information about hotline events and designing a graph construction scheme to construct an event knowledge graph based on historical event texts.

2. Performing subgraph retrieval based on three types of information obtained from event extraction module and designing a subgraph fusion scheme to fuse the three types of subgraph retrieval results to obtain the final retrieved subgraph.

3. Fusing the graph-based prior knowledge extracted by fused-GCN and BERT with the “sanding” prior knowledge and using the KBQA-based approach to predict the probability distribution of the departments, and finally ranking the distribution to output the optimal assignment department.

Related Work

Government Hotline

In order to realize automatic hotline event classification, methods based on machine learning are generally adopted (Chen and Guestrin 2016). Firstly, the features are extracted from the text description of a hotline event, then multiple features are connected to form a high-dimensional feature vector, and finally the classifier is used to complete the classification. These methods require a lot of feature engineering. Selection and analysis of these features are complex which need to spend a lot of energy to conceive. In order to solve the problem that single neural network (e.g., CNN, RNN and LSTM) is difficult to discover the tiny difference among Chinese hotline events, Chen proposed event classification method fusing RoBERTa and feature extraction (Chen et al. 2022). The method obtains context-aware semantic feature vectors from textual descriptions of a work-order by RoBERTa and uses a hybrid neural network combined with CNN and BiGRU to explore the local and global features of the work-order semantic encodings. The fused features are sent into the classifier to finish work-order classification. Due to the complexity of events, it is difficult to accurately assign events by simply using the text corpus of 12,345 government hotline. In follow-up researches, the public data of government departments such as department description or “sanding” (containing the description of department responsibilities, organization settings and staffing) is also often fed into the task as additional knowledge to strengthen the correlation between events and government and improve the assignment accuracy of hotline events. Liu proposed a multi-label classification framework based on GCN, BERT, and memory network for Chinese government hotline (Liu 2022). The framework constructs the event graph with the abstract meaning representation and extracts the event topic information vector with GCN, and uses the label count-based multi-label selection to sort the event label candidate set and guide to output the optimal multi-label set of the event contained in Chinese government hotline. However, it cannot perform event assignment. She proposed a joint learning method for event text classification and event assignment for Chinese government hotline (She, Chen, and Chen 2022). The method uses GCN and BERT to process the event text of the Chinese government hotline, and the result vector is fused to achieve the event classification task. At the same time, the attention mechanism is used to process the GCN vector and BERT vector, and complete the event allocation based on the assignment record in the historical event database. However, the paper only used keywords in sentences as entities when performing graph construction, without considering the significance of the keywords for event assignment, as well as not defining relationships based on assignment, and only used whether different entities appeared in the same sentence as associated edges. Based on the event extraction method (Chen 2022), Chen proposed an automatic event assignment method for Chinese government hotline (Chen et al. 2022). The method uses the CRF network to solve the problem of sequence annotation for event texts and design an external knowledge embedding layer which integrates “rights and responsibilities lists” with the historical information of the event to assist assignment. However, the method cannot use historical information to enhance the accuracy of event assignment.

Event Extraction

Event extraction is an important part of many downstream applications such as information retrieval, Question-Answer (QA) systems, and decision support, so this field has been a hot spot in natural language processing research. Traditional event extraction methods are mostly based on manual experience, but when dealing with unstructured text, it is difficult for such methods to mine the deep semantic features in it. Notably, deep learning models such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs) have shown considerable advantages in capturing complex event patterns in unstructured text data (Lyu et al. 2021). Chen et al. introduced a word representation model DMCNN in (Chen et al. 2015). This method captures meaningful semantic rules for words and uses a CNN-based framework to capture semantic features at the sentence level. Similarly, JRNN (Nguyen, Cho, and Grishman 2016) is a joint-paradigm based bidirectional recurrent neural network for event extraction, which consists of encoding phase and prediction phase using RNN to summarize contextual information. However, while such methods can capture local semantic information of text to some extent, they are still inadequate in global semantic capture. With the proposal of the attention mechanism, the number of models that incorporate the attention mechanism has gradually increased, such as joint multiple event extraction (JMEE) (Liu, Luo, and Huang 2018) which uses attention-based graph convolutional networks for joint modeling to extract multiple event trigger words and thesis elements, and optimizes the biased loss function when jointly extracting the event trigger words and thesis elements in order to solve the dataset imbalance problem. For complex situations in real-world scenarios, research has been conducted to enhance the corresponding extraction effect using various techniques, such as using grammatical relations between words to improve the efficiency of event extraction (Cao et al. 2022; Li et al. 2022), or optimizing the effect of event extraction on a less-sample dataset based on syntactic structural information (Fei et al. 2022, 15460–75). Pre-trained language models such as BERT with other BERT-based derivatives have become effective tools for natural language understanding tasks including event extraction (Fincke et al. 2022; Liu et al. 2022). These models utilize large amounts of unlabeled textual data to learn rich contextual representations and can be fine-tuned for specific downstream tasks. Yang et al (2019) proposed a pre-trained language model-based event extractor (PLMEE) framework, which automatically generates labeled data through “editing prototypes” and filters development samples according to quality ranking. Fei et al. proposed a method combining large-scale biomedical knowledge graph to enrich the contextual language model and improve the effect of biomedical information extraction (Fei et al. 2022, bbaa110). The pre-trained language model effectively improves the accuracy of the extracted semantic information, but in the real-world scenario of hotline event assignment, it often faces comprehension bias due to the lack of external knowledge.

KBQA

Question Answering (QA) is an important research area in artificial intelligence and natural language processing. Its goal is to enable computers to understand the questions asked by users and give accurate answers. According to the different data sources, technical methods, and application scenarios it uses, it can be categorized into different ways, such as textual QA systems (QA) that use natural language processing techniques to retrieve answers from textual data or to generate answers based on textual data, and visual QA systems(VQA) that use computer vision techniques to process image and video content and combine them with techniques of natural language processing, information retrieval (Li et al. 2023), etc., to textually generate answers to questions (Cao and Li 2023a, 2023b). These question answering systems usually generate answers based on input text, images, and video content, whereas knowledge-based QA (KBQA) incorporates prior knowledge from a knowledge base to generate more accurate answers.

KBQA aims to answer a question over a knowledge base (KB) that is a structured database containing a collection of facts (alias triples) in the form (subject, relation, object). According to different execution paradigms, KBQA can be divided into two categories: the Semantic parsing-based (SP-based) method and Information retrieval-based (IR-based) method (Lan et al. 2021). An essential step is getting a question representation as accurate as possible in KBQA. To represent long-range dependencies between the answer and the topic entity in question, Luo et al. (2018) extracted the dependency path between them. By encoding the directional dependency path, they concatenated both syntactic and local semantic features to form a global question representation. In order to alleviate the inaccurate syntactic parsing of complex questions, Sun et al. (2020) leveraged the skeleton-based parsing to obtain the trunk of a complex question, which is a simple question with several branches (i.e., head word of original text spans) to be expanded. Under such a skeleton structure, it is more accessible to obtain accurate parsing results only by further analyzing simple sentences. It is also crucial to analyze the syntax of queries and ensure that the generated logic forms can meet the complex syntax of questions. But this method has a greater likelihood of poorer accuracy for QA in complex language scenarios. Maheshwari et al. (2019) proposed a novel ranking model which exploits the structure of query graphs and uses attention weights to explicitly compare predicates with natural language questions. Specifically, they proposed a fine-grained slot matching mechanism to conduct hop wise semantic matching between the question and each predicate in the core reasoning chain. Instead of capturing semantic correlations between a question and a simple relation chain, Zhu, Cheng, and Su (2020) focused on structure properties of query and conducted KBQA with query-question matching. They employed a structure-aware encoder to model entity or relation context in a query, promoting the matching between queries and questions. Instead of enumerating logic forms with a single pass, researchers try to propose methods to generate the complex queries with multiple steps. However, this method has very high requirements for retrieval accuracy and efficiency. Wei et al. (2018) and Nikita, Xinyi, and Jagadish (2019) proposed to first decompose a complex question into multiple simple questions, where each simple question was parsed into a simple logic form. The final answers are obtained with either the conjunction or composition of the partial logic forms. This decompose-execute-join strategy can effectively narrow down the search space. Unlike decomposing a complex question to sub-questions, a number of studies adopted the expand-and-rank strategies to reduce the search space by expanding the logic forms in an iterative manner. Yunshi and Jing (2020) defined three expansion actions for each iteration, which are extending, connecting, and aggregating to correspond to multi-hop reasoning, constrained relations, and numerical operations, respectively. To solve the problem of missing graph structure in the IR-based method, Haitian et al. (2018) proposed to complement the graph with extra question-relevant sentences as nodes and reason on the augmented heterogeneous graph. According to the entities mentioned in sentences, they linked them to corresponding entities on the graph and viewed them as nodes. Most of the current KBQA is offline training and online deployment. However, due to the unbalanced data distribution and insufficient amount during the training process, it is impossible to learn the unsolvable problems that appear online in the actual situation. Two research directions will be how to solve the imbalance of data and the processing of unseen data online.

Methodology

We proposed a KBQA-based method to obtain the corresponding department of the event for event assignment task, which can effectively use the event extraction results and the knowledge graph built based on historical events, and use the QA method to achieve event assignment. The event assignment model in this paper mainly consists of three parts: event association information retrieval module, event association feature extraction module and event allocation department prediction, and the specific model structure is shown in Figure 1. For a given input event text, we first used event extraction algorithm to obtain event trigger words, event types, and argument roles, then carried out subgraph retrieval to get subgraph, and used Fusion-GCN model to obtain subgraph representation features. Secondly, we use GloVe-based text retrieval method to obtain the most relevant “sanding” text, and use BERT to obtain the “sanding” representation vector. Then, we use another BERT to extract the event text features and generate the fused event vector set by combining the set of departmental entities in the subgraph. Finally, the fused event vector set elements vector, graph representation features vector, and the “sanding” representation vector are fused to predict the department scores, and the optimal assignment department is output according to the score results.

Figure 1. Model structure.

Event Extraction

The input of the event extraction module for historical events is a complete event description text, and the output of the module is the corresponding event trigger words, event type, and event argument roles. To make full use of the contextual relationships in the event description text and the key information contained therein, this paper uses a joint-based model for event extraction. Considering that argument roles are more dependent on the key information in the event description text (Ding et al. 2019), the event extraction module has argument roles extraction as an end task. The input of this task depends on the extraction results of event trigger words and event types, and the three tasks are jointly optimized as a whole module. The specific structure is shown in Figure 2. First, the event description text is sent into the trigger extraction layer to generate a list of trigger words for the event, and then the event trigger words embedding vectors are input into the event classification layer to output the corresponding event type prediction results, and finally the event classification result encoding vector is input into the argument roles extraction layer together with the trigger words embedding vectors and words embedding vectors to generate the corresponding event argument roles prediction results.

Figure 2. Event extraction module structure.

Considering the fact that Chinese has more semantic levels and contains more complex information, we firstly perform word segmentation on the event text before processing. Specifically, the Chinese word segmentation tool is used to divide the input text into word sequence$\mathit{X}=\{x_1,x_2,\dots ,x_n\}$ (where $\mathit{n}$ is the length of the word sequence obtained by word segmentation tool). The content of the description of government hotline events often involves government entities, which are often composed of multiple juxtaposed phrases, and the noun as a determiner modifies the noun, which makes the length of the entity larger, and the difficulty of entity boundary identification is relatively high, e.g., “芜湖市人民政府国有资产监督管理委员会” (SASAC of Wuhu Municipal People’s Government). On the other hand, the vocabulary that constitutes the governmental entities has obvious official text characteristics; the vocabulary that cannot constitute the governmental entities also has significant textual characteristics. In order to make full use of the domain characteristics of the constituent words of governmental entities and improve the recognition accuracy of governmental entities, we use “standard phrase dictionaries” and “nonstandard phrase dictionaries” to adjust the segmentation results. The “standard phrase dictionary” consists of high-frequency words that are involved in the formation of governmental entities, while the “nonstandard phrase dictionary” consists of high-frequency words that are difficult to form governmental entities. When performing word segmentation, the results are compared with the positive and negative lexicon, and sequences that satisfy the positive words are merged, while sequences that satisfy the negative words are split. Then the word2vec model is used to encoding word sequence to obtain the initialized semantic encoding vector$\mathit{Init}=\{\mathit{ini}{t}_1,\mathit{ini}{t}_2,\dots ,\mathit{ini}{t}_n\}$. Meanwhile, for each word, the corresponding position vector $\mathit{po}{s}_i$and segment vector $\mathit{segmen}{t}_i$are constructed, both of which are one-dimensional vectors with the length of the word sequence, where the position vector $\mathit{po}{s}_i$ is set to 1 at the corresponding position of the word and 0 at the remaining positions, while the segment vector is used as the text vector, with 1 in the position corresponding to the sentence in which the word is to be encoded and 0 in the rest. After that, the position vector and the segment vector are input into the pre-trained language model together with the initialization embedding vector to obtain the corresponding semantic embedding vector:

(1) $e_i=\mathit{LM}(\mathit{ini}{t}_i,\mathit{po}{s}_i,\mathit{segmen}{t}_i)$(1)

The semantic encoding vectors constitute the set $\mathit{E}=\{e_1,e_2,\dots ,e_n\}$. To obtain the trigger words in the text, the trigger word extraction task is converted into a binary classification problem of whether the words in the word sequence are trigger words or not. A classification model is constructed and the word embedding vectors $\mathit{E}$ in are input to the classification successively to obtain the trigger-or-not probability distribution:

(2) $P_i^{tri}=\mathit{softmax}({\mathbf{W}}_{\mathit{tri}}{e}_i+b_{\mathit{tri}})$(2)

where ${\mathbf{W}}_{\mathit{tri}}$,${\mathrm{b}}_{\mathit{tri}}$ are the learnable parameters. After the classification probability distribution of the sequence is obtained, the word sequence corresponding to top-n in the probability distribution is selected as the final trigger word prediction result. For the trigger extraction layer, cross-entropy loss function is used to optimize this layer:

(3) $L_{\mathit{tri}}=-[y_i^{tri}\mathit{log}{\hat{y}}_{\mathit{i}}^{\mathit{tri}}+(1-y_i^{tri})\mathit{log}(1-{\hat{y}}_{\mathit{i}}^{\mathit{tri}})]$(3)

where $y_i^{tri}$ denotes the actual trigger classification result and ${\hat{y}}_{\mathit{i}}^{\mathit{tri}}$ denotes the predicted trigger classification result.

After completing the trigger extraction, the sequence of trigger words $\mathit{T}=\{t_1,t_2,\dots ,t_k\}$ corresponding to the event text is generated. Then, the semantic vector corresponding to the trigger words in the sequence is fused, and then input into the multi-classification network to obtain the corresponding probability distribution of event types $P_i^{\mathit{type}}$, and the highest probability result is selected as the prediction result of the event types.

(4) $P_i^{\mathit{type}}=\mathit{softmax}({\mathbf{W}}_{\mathit{type}}{e}^{\prime }+{\mathrm{b}}_{\mathit{type}})$(4)

(5) $e^{\prime }=max(e_{t_1},e_{t_2},\dots ,e_{t_k})$(5)

where ${\mathbf{W}}_{\mathit{type}}$,${\mathrm{b}}_{\mathit{type}}$ is the learnable parameter, $e_{t_k}$ indicates semantic embedding vector and $max$ denotes the maximum pooling function, taking the maximum value in each dimension respectively. The event classification layer is optimized using multi-class cross-entropy function:

(6) $L_{\mathit{type}}=-\frac{1}{J}\sum _i\sum _{\mathit{c}=1}^{\mathit{LC}}{y}_{\mathit{ic}}\mathit{log}(p_{\mathit{ic}})$(6)

where $\mathit{LC}$ is the number of event types, $y_{\mathit{ic}}$ is symbolic function that takes 1 if the true event type of the sample event is$\mathit{c}$ and 0 otherwise. $p_{\mathit{ic}}$ denotes the prediction probability of sample event$\mathit{i}$as type$\mathit{c}$.

The argument roles extractor layer aims to extract arguments and their roles given the trigger words. Considering the high dependence of arguments on the key information carried by the trigger words and the fact that most of the arguments are noun phrases, this paper use a multi-classification to predict the argument roles corresponding to the word sequence of event description text. Meanwhile, since the event set contains many different event types, the classification network contains the roles corresponding to multiple event types in order to make full use of the event type information. After the trigger extraction layer, the key information position vector $\mathit{po}{s}_{\mathit{tri}}$ is constructed based on the result of the trigger words sequence, and the key information position vector is a one-dimensional vector, which is marked as 1 at the word position corresponding to the trigger words and 0 at the rest positions to inform the encoding model where the key information is located. Then, the words embedding vectors $\mathit{E}$ are input to the pre-trained language model together with the position vectors $\mathit{pos}$ and key information position vectors $\mathit{po}{s}_{\mathit{tri}}$ to obtain embedding vectors $\mathit{e}{{\mathit{ }}^{\mathrm{\prime }}}_{\mathit{i}}$.

(7) $\mathit{e}{{\mathit{ }}^{\mathrm{\prime }}}_{\mathit{i}}=\mathit{LM}(\mathit{ini}{t}_i,\mathit{po}{s}_i,\mathit{po}{s}_{\mathit{tri}})$(7)

Then the embedding vectors are fed into the argument roles classifier together with the event type embedding vector $q_{\mathit{type}}$ to generate the probability distribution of argument roles:

(8) $P_i^{rol{e}_j}=\mathit{softmax}({\mathbf{W}}_{\mathit{role}}\left(e^{{ }^{\prime \prime }}\oplus q_{\mathit{type}}\right)+{\mathrm{b}}_{\mathit{role}})$(8)

where ${\mathbf{W}}_{\mathit{role}}$,${\mathrm{b}}_{\mathit{role}}$ are learnable parameters, Event type embedding vector $q_{\mathit{type}}$ is a one-dimensional vector constructed by the generated results of the event type classification layer, and its length is the number of event types and 1 is set on the position corresponding to the predicted event type result, and 0 is set on the rest positions. After the probability distribution is obtained, the top-2 probability is selected for comparison. If the predicted probabilities of the first two argument roles are similar, then the two prediction results will be output at the same time; otherwise, only the prediction result with the highest probability will be output. The element role extraction module is optimized by multi-classification cross entropy function:

(9) $L_{\mathit{role}}=-\frac{1}{N}\sum _i\sum _{\mathit{c}=1}^M{y}_{\mathit{ic}}log(p_{\mathit{ic}})$(9)

where $\mathit{M}$ is the number of argument roles and $y_{\mathit{ic}}$ is a symbolic function that takes 1 if the true roles of sample $\mathit{i}$ is $\mathit{c}$ and 0 otherwise. $p_{\mathit{ic}}$ denotes the probability of predicting the outcome argument role$\mathit{c}$ for sample$\mathit{i}$.

For the overall event extraction module, the joint function of three loss functions is used as the loss function of event extraction module:

(10) $\mathit{Loss}=L_{\mathit{tri}}+L_{\mathit{type}}+L_{\mathit{role}}$(10)

Knowledge Graph Construction

When constructing a knowledge graph, in order to ensure accurate and efficient extraction of key information from complex event descriptions and to gradually improve the graph structure, the model in this paper adopts a bottom-up construction method. This method starts from specific events, gradually abstracts and generalizes, and finally forms a comprehensive and fine knowledge system. Through the event extraction module, three key information including trigger words, event types, and argument roles is extracted from the event description text, and in the knowledge graph construction module, we construct the corresponding event knowledge graph in a bottom-up manner. The bottom-up knowledge graph construction method is an iterative update process, and each round of update includes two parts: information extraction and knowledge fusion. The specific process for each step in the iteration is described next.

Information Extraction

As the data source for the construction of event knowledge graph, the process of information extraction in this paper includes two parts: entity construction and relationship extraction.

Specifically, the entity construction part mainly consists of the results of the event extraction module, but in addition to the event trigger words, event types, and argument roles obtained from the event extraction module, this paper also uses the event ID information and assignment department information of each historical event as the other two source of entity information needed to construct the knowledge graph.

In the relationship extraction part, the entity information obtained from the entity extraction module is associated by constructing the graph relational model to generate the basic triples that build the event knowledge graph. Relational schema construction establishes association schemas among entities to standardize the final output of fact representation patterns from information extraction, while the database generated by schema relationship construction can ensure less redundancy of relationships. The event knowledge graph in this paper includes five types of entities, and according to the cognitive framework of the basic elements structure of events based on prior knowledge, the corresponding knowledge model framework of these five types of entities is constructed, and the relationship schema diagram is shown in Figure 3.

Figure 3. The relationship schema diagram.

This knowledge mapping relationship pattern graph contains 5 types of entities and 4 types of relations. Considering that there is no significant ”entity-attribute” attribution relationship among the elements extracted in the event extraction module, multiple entity nodes in each type of entity are directly associated with events. Due to the diversity of event types, there are many ways to describe logical relationship between multiple argument roles or multiple trigger words and the same event, and in order to unify the descriptions, a unified relationship description is used between multiple argument roles or multiple trigger words and the same event. Take the event ”(3471, ‘上周日，我在某车站预约的车牌号ABCDE的出租车司机直接拒载 Last Sunday, the cab driver with the license plate number ABCDE I booked at a station refused to take me directly),’ ‘公安局 (Public Security Bureau)’” as an example, according to this event text, the examples of relational triples formed are shown in Table 1.

Table 1. Examples of relational triples.

Entity 1	Relation	Entity 2
3471	Contains	拒载 (refuse to take passenger)(Trigger)
3471	Belong to	出租车拒载 (Cab refusal)(Event type)
3471	Include	上周日 (Last Sunday)(Argument role)
3471	Include	车牌号 ABCDE （论元信息）
3471	Dealt by	公安局
3471	Contains	拒载 (refuse to take passenger)(Trigger)

According to the results shown in Table 1, after completing the above pattern construction, the extraction results around the same event will form n + 1 triples around the event entity, where n indicates the number of key elements obtained by the event extraction module.

Knowledge Fusion

After the information extraction completes the extraction and collation, the corresponding entities, relations, and corresponding triples are output, but these triples are independent of each other and do not form an association relationship, which still cannot form a hierarchical and logical knowledge graph. Meanwhile, considering that the extracted entity and relationship information may have redundant and wrong information, the knowledge fusion is used to integrate the entity, relationship, and triad information. The knowledge fusion mainly includes two parts: entity link and knowledge merge. The main processing flow of the knowledge fusion is described as shown in Algorithm 1.

Table

Algorithm 1: Knowledge Fusion

Input: Entity Set E:{entity1,entity2, … ,entitym}, Role Set R:{role1, role2, … , rolen} Output: Aligned Entity Set E’

group←K-Means(R) // Iterate over each entity in the entity set E for i in (1, m):  // Iterate over all entities except the current i  for j in (i+1, m):   if str(entityi)==str(entityj):    // Calculate the similarity of two entities    sim1=Sim(Concat(entityi, contexti),Concat(entityj, contextj))    // If the similarity is less than threshold δ, the two entities are merged    if sim1≥δ:     entityi←entityj    // If the similarity of the two entities is less than the threshold δ, but the similarity in the sentence containing    contextual information is greater than ζ, the two entities are merged    elseif (entityi∈R) &; (entityj∈R):     labeli=groupi     labelj=groupj     if labeli==labelj:       sim2=Sim’(Concat(entityi, senti), Concat(entityj, sentj))       if sim2≥ζ:       entityi←entityj

Entity link mainly corrects and aligns the redundant referents or ambiguous referents existing between entities, so that a part of the constructed triples forms a valid association relationship. Entity link is mainly containing two key elements: entity disambiguation, which eliminates homonymous ambiguity, and co-reference disambiguation, which eliminates different referents to the same object. The entity disambiguation part for any two entities composed of the same name entity pairs in the entity collection obtained from information extraction, the two entities, and their surrounding words are associated, and then input into the pre-trained language model to obtain the corresponding embedding vector, after which the semantic similarity between the two entities is calculated, merging the entities whose similarity is greater than a certain threshold and distinguishing the entities with the same name that have different references (Mueller and Thyagarajan 2016):

(11) $\mathit{G}(e_i,e_j)=\sum e_i,e_j-\frac{\sum e_i\sum e_j}{\mathit{d}}$(11)

(12) $\mathit{Sim}(E_i,E_j)=\frac{G(E_i,E_j)}{\sqrt{\mathit{G}(E_i,E_i)\mathit{G}(E_j,E_j)}}$(12)

where $E_i$ denotes the embedding vector of entity $\mathit{i}$ and $\mathit{d}$ denotes the length of the entity embedding vector.

Since the co-reference disambiguation occurs mainly in the argument roles, thus this paper only use co-reference disambiguation on the entities set of the event argument roles. Considering that the co-reference disambiguation between argument roles is closely related to the context of event description text, after obtaining the set of argument role entities, the semantic embedding vector of the argument role and its sentence are fused to form the fusion embedding vector of argument role. After that, the fusion embedding vectors are clustered, and the fusion vector similarity is calculated in the set of argument roles embedding vectors aggregated in the same cluster, and the pairs of arguments with similarity greater than a certain threshold are merged to form a unified entity referent.

(13) $S_{rol{e}_i}=\mathit{\mu }{e}_{rol{e}_i}+(1-\mathit{\mu })e_{\mathit{sent}}$(13)

(14) $\mathit{Rel}(\mathit{rol}{e}_m,\mathit{rol}{e}_n)=\mathit{Sim}(S_{rol{e}_m},S_{rol{e}_n})$(14)

where $S_{rol{e}_i}$ denotes the fusion embedding of the argument role $\mathit{i}$, $e_{rol{e}_i}$,$e_{\mathit{sent}}$ are the embedding vectors of the argument role and its sentence, and $\mathit{\mu }$ is the learnable weight.

After the completion of entity disambiguation and co-reference disambiguation, the entity information of triples has been integrated and aligned, and then n subgraphs $\{g_1,g_2,\dots ,g_n\}$ are constructed from these triples generated from each event. Then, the duplicate entities are fused and the connections between the subgraphs are established to form the complete event knowledge graph.

KBQA-Based Event Assignment

Event Information Retrieval

We use two parallel pipelines to obtain event-related features: one is subgraph retrieval based on the knowledge graph, i.e., retrieval on the knowledge graph, which returns the set of entities; the other is text retrieval based on the “sanding” responsibilities, i.e., text retrieval on the “sanding” text corpus, which returns the document set. The retrieved entities and documents are then fed into the prediction model as fusion information of events.

Subgraph Retrieval

In order to retrieve entities related to input events from the knowledge graph, we first perform event extraction on the input event text and obtained three types of entities, namely event trigger word, event type, and argument role, as seed entities, denoted as set $\{S_e\}$. Then, the Personalized PageRank (PPR) algorithm (Taher 2002) is applied to the elements in the set $\{S_e\}$ to identify entities that may be related to the event. For the entity identified by the PPR algorithm, the inner product method is adopted to calculate the score of the entity word vector and event sentence vector, and the score is taken as the edge weight of the entity and the seed entity.

Through this method, we can first obtain the subgraphs of three different types of entities associated with trigger words, event types, and argument roles, and merge the same entities in the subgraphs of the same entities (trigger words, event types and event roles) to form three different type subgraphs. Therefore, we also adopt the same method to merge the same entities in the subgraph of different types to form the final subgraph.

Specifically, we first merge the same entities in n subgraphs of n seed entities of the same type to form three subgraphs (G₁, G₂ and G₃) of the same type. Then the same entities in three subgraphs of the same type are merged to get the final subgraph G. For these merged entities, we trace back to the root (i.e., the seed entity) in each subgraph and trace forward to the leaves. We then retain only the entities and relations on all trace path to form the subgraph. As shown in Figure 4, given the input event text “车牌号ABCDE的出租车司机直接拒载 (the cab driver with the license plate number ABCDE refused to take a passenger directly),” the event extraction algorithm is used to obtain the seed entities “拒载,” “出租车拒载” and “车牌号 ABCDE.” And then we use the seed entities to retrieve three types of subgraphs {G₁, G₂, G₃}, and merge the same entities (the entities in red boxes in Figure 4) to obtain the final subgraph G.

Figure 4. Subgraph retrieval procedure.

The subgraphs G₁, G₂, and G₃, derived from the three entity types related to events (event trigger words, event types, and argument roles), along with the final subgraph G obtained through subgraph merging, are input into the Subgraph Reader module to extract subgraph features.

Document Retrieval

We used 355 “sanding” text data as the document retrieval corpus, and carried out sentence-level document retrieval, i.e., we retrieved each sentence in the “sanding” retrieval corpus. The execution steps of our “sanding” document retrieval are as follows:

Firstly, we vectorize all sentences in the “sanding” retrieval corpus using GloVe word table to obtain a vector representation set of all sentences, denoted as $\{V_i^{sa\mathrm{ndin}}\},\mathit{i}=1,2,\dots$.

Secondly, we vectorize the input event text with the same GloVe word table to obtain the event text representation vector V_e, and calculate the inner product of V_e between the elements in $\{V_i^{sa\mathrm{ndin}}\}$, then sort all the sentences according to the inner product result to get top-5 associated sentences.

Finally, trace back to the source according to the top-5 associated sentences, the scores of the “sanding” text corresponding to the top-5 sentences are calculated, and the “sanding” text with the highest score is selected as the text retrieval result.

The “sanding” original text calculated by the aforementioned Text Retrieval algorithm is used as one of the inputs for the Text Reader module to extract “sanding” features.

Event Feature Extraction

Subgraph Reader

For the constructed subgraphs G₁, G₂, G₃, and G, we use Fusion-GCN (Kipf and Welling 2019) for subgraph feature extraction, which uses multi-layer residual fusion for single-graph feature extraction on the basis of GCN, and uses attention mechanism for fusion of multi-graph features to accumulate knowledge of multiple subgraphs.

To extract the graph representation features of each key vertex (trigger word, event type, argument role, and department), we pair G₁, G₂, G₃, and G respectively to form pairs, namely (G₁, G), (G₂, G), and (G₃, G). For the key input vertex V, we define graph convolution operation in two vertex domains to aggregate the local and global features of this vertex. This graph convolution operation for feature extraction is calculated as follows:

(15) $H^{(\mathit{l})}=\left\{\begin{array}{cc}Re\mathrm{LU}(\tilde{A}(H^{(\mathit{l}-1)}+H^{(\mathit{l}-2)})W^{(\mathit{l})}+b^{(\mathit{l})}),& \mathit{l}>1\\ Re\mathrm{LU}(\tilde{A}{H}^{(\mathit{l}-1)}{W}^{(\mathit{l})}+b^{(\mathit{l})}),& \mathit{l}=1\end{array}\right.$(15)

where $\tilde{A}=D^{-\frac{1}{2}}\mathit{A}{ }^{\mathrm{\prime }}{D}^{-\frac{1}{2}}$, $\mathit{A}{ }^{\mathrm{\prime }}=\mathit{A}+\mathit{I}$, A is the adjacency matrix of the input graph, I is the identity matrix, $D_{\mathit{ii}}={\sum }_j{\mathit{A}{\mathit{ }}^{\mathrm{\prime }}}_{\mathit{ij}}$, $W^{(\mathit{l})}$ and $b^{(\mathit{l})}$ are learnable parameters, $H^{(0)}=\mathit{V}$. To further preserve the graph feature information, we fuse the graph convolution output using a multi-layer aggregation, which is calculated as shown below:

(16) ${\mathit{H}}^{(\mathit{out})}={\lambda }_{1}max({\mathit{H}}^{(1)},{\mathit{H}}^{(2)},\dots ,{\mathit{H}}^{(\mathit{L})})+{\lambda }_2{\mathit{H}}^{(\mathit{L})}$(16)

where${\lambda }_1+{\lambda }_2=1$, L is the number of graph convolution layers, in this paper we set L = 3.

According to the above formula, the vertex representation vector set of each subgraph can be calculated, and the representation vector of three different subgraphs can be obtained by weighted fusion of its seed entity vectors. Different types of subgraph representation vectors and final subgraph representation vector constitute graph vector pair $({H_{G_i}}^{(\mathit{out})},{H_G}^{(\mathit{out})})$, where ${H_{G_i}}^{(\mathit{out})}$ represents the local feature representation vector and ${H_G}^{(\mathit{out})}$ represents the global feature representation vector. Then two vectors are fused using the attention mechanism to get the fusion representation vector $V_{G_i}$:

(17) $V_{G_i}=\mathrm{softmax}(\frac{W_1^{\hspace{0.17em}}{H}_G^{(out)}{(W_1^{\hspace{0.17em}}{H}_{G_i}^{(\mathit{out})})}^{\mathrm{T}}}{\sqrt{\mathit{d}}})W_2^{\hspace{0.17em}}{H}_{G_i}^{(\mathit{out})}$(17)

where $W_1^{\hspace{0.17em}}$, $W_2^{\hspace{0.17em}}$ are learnable parameters, d is the number of attention heads.

According to the above formula, we calculated the representation vectors of three types: trigger, event type, and argument role respectively, and then concatenate them to obtain the final graph representation vector $V_G=\mathrm{concat}(V_{G_1};V_{G_2};V_{G_3})$. And all the department vertex representation vector of the final subgraph, denoted as $\{V_i^d\},\mathit{i}=1,2,3,\dots$.

Text Reader

The text reader consists of two parts: the problem text representation (i.e., the vectorized representation of the input event text) and the “sanding” text representation.

For the problem text representation, we fuse the input event text with the elements in the department vertex representation vector set $\{V_i^d\}$ as the problem representation vector. The specific way is as follows, A BERT is used to extract feature from input event text and get event representation vector $V_0^q$:

(18) $V_0^q=\mathit{f}(\mathit{q})$(18)

where q is the input event text and f is the BERT pre-trained model. The vector $V_0^q$ is fused with each element in the department vertex representation vector set $\{V_i^d\}$ to obtain the fused problem vector set $\{V_i^q\},\mathit{i}=1,2,3,\dots$.

(19) $V_i^q={\gamma }^i{V}_0^q+{\phi }^i\mathit{tanh}(W_i^q[V_0^q,V_i^d,V_0^q-V_i^d])$(19)

(20) ${\gamma }^i=\mathrm{sigmoid}(\tilde{W}\mathit{iq}[V_0^q,V_i^d,V_0^q-V_i^d])$(20)

where $W_i^q$ and $\tilde{W}\mathit{iq}$ are learnable parameters, and ${\gamma }^i+{\phi }^i=1$.

For the retrieved “sanding” text, we use another BERT model to extract features directly to obtain the “sanding” feature representation vector V_sanding,

(21) $V_{sa\mathrm{nding}}^{\hspace{0.17em}}=\hat{f}(T_{\mathit{sanding}})$(21)

where, $\hat{f}$ is the BERT pre-trained model.

Event Assignment Department Prediction

After we get the event knowledge graph feature, the “sanding” feature and the input event feature, the graph representation vector $V_G$ is concatenated with the “sanding” representation vector V_sanding, and then the probability prediction of the department entity is made by matching the problem vector $V_i^q$ and the concatenate vector:

(22) $p_i=\mathrm{softmax}((V_i^q{)}^{\mathrm{T}}{W}_i[V_g;V_{\mathit{sanding}}]+b_i)$(22)

where, $W_i$ and $b_i$ are learnable parameters. In this paper, we use the binary cross-entropy as the training loss of the model and use the final output probability as the score of the department.

All elements in the vector set $\{V_i^q\}$ are predicted to obtain the score set $\{S_i^{\hspace{0.17em}}\},\mathit{i}=1,2,3,\dots$ of the departments. Finally, set $\{S_i^{\hspace{0.17em}}\}$ is sorted and the optimal department is selected as the event assignment department.

Experiment

Experiment Settings

Dataset

Table 2. Dataset description.

Dataset	Documents	Sentences	Characters	Department	Avg length	Max length	Min Length
Dataset1	83073	214638	9306k	30	112.02	500	10
Dataset2	106926	276036	11962k	36	111.87	500	10

We construct two experimental datasets based on the real government hotline event assignment result. After counting on the full event dataset, more than 98% of the data are less than 300 words and 0.32% of the data are less than 10 words, and most of this data are wrong calls without proper semantic information, so in the pre-processing stage of the data, we removed the data with more than 500 words and less than 10 words. Firstly, the data was preprocessed to delete the events of more than 500 words and less than 10 words. After processing, the Dataset1 contains 27,691 events of 30 municipal departments, and the Dataset2 contains 35,642 events of 36 municipal departments. Secondly, based on the real event assignment results of the government hotline 27,691 of Dataset1 and 35,642 of Dataset2 positive sample event data corresponding to “event-department” were constructed respectively. Finally, 27691 positive samples (matched) were randomly selected from 29 unmatched departments to construct 55,382 negative samples (unmatched) to generate Dataset1. 35642 positive samples (matched) were randomly selected from 35 unmatched departments to construct 71,284 negative samples (unmatched) to generate Dataset2. The description of the data set is shown in Table 2.

Hyperparameter

In the Event Extraction section, BERT-wwm-ext is used as pre-trained language model in the Event Extraction module as the base encoder for the Trigger Word Classifier and the Argumentative Role Classifier to encode the input hotline event text. Meanwhile, Adam optimizer with learning rate of 10e-5 is used as the model optimizer, the training epoch is set to 30 and the batch size is set to 32. In the KBQA-based Event Assignment section, for the GloVe model used in the Event Information Retrieval module, the historical case data is used for training to obtain the vector dimension of 128. We also use BERT-wwm-ext as pre-trained language model to process the text retrieval results and the input hotline event text. In Fusion-GCN, GCN uses a 3-layer graph convolution operation to obtain the corresponding graph feature vectors. We use Adam optimizer to train the model with 20 epochs, batch size 16 and learning rate 10–5.

Evaluation Metrics

We evaluate the performance of the subgraph retriever by the coverage rate of the departments corresponding to the events. We calculate the proportion of the subgraph retrieval results containing the real processing departments of the events, which is calculated as follows:

(23) $\mathit{coverage}\mathit{\hspace{0.17em}}\mathit{rate}=\frac{N_{sg}^{gt}}{N_{\mathit{sg}}}$(23)

where $N_{sg}^{gt}$ represents the number of subgraphs containing the actual assignment department of the event and $N_{\mathit{sg}}$ represents the total number of retrieval subgraphs. Since the final assignment department is generated from the subgraph retrieval results, this metric reflects the upper bound of the model performance.

For the overall model’s assignment performance, we use Hits@1 to evaluate whether the top-1 predicted answer is correct. Meanwhile, we further evaluated the model effect by calculating the Precision, Recall, and F1 values of the model.

Baselines

We compare the model with three types of baseline models on the constructed dataset:

1. Text classification based model, i.e., using text classification models to directly process event text for event assignment, including: HAN (Yang et al. 2016), XLNet (Yang et al. 2019).

2. Answer selection based model, i.e., using event text as the question and “sanding” text as the answer to perform answer selection for event assignment, including: BERT-BiGRU-based (Nikita, Xinyi, and Jagadish 2019), EEuEK (Chen 2022), BGMA (Devlin et al. 2019), DeBERTa-SSP (Liello et al. 2022).

3. KBQA based model, i.e., building knowledge graphs, using event text as questions, and using KBQA method for event assignment, including: Pullnet (Haitian, Tania, and William 2019), EmbedKGQA (Apoorv, Aditay, and Partha 2020), RnG-KBQA (Ye et al. 2022), UniK-QA (Oguz 2022), SKP (Dong et al. 2023).

Comparisons of Baselines

In order to verify the effectiveness of the model in this paper, we compared the model with the baseline models on the constructed dataset, and the experimental results are shown in Table 3, where “no-sanding” denotes the model with text retrieval and “sanding” text embedding removed based on our model, and only the knowledge graph features are used for event assignment.

Table 3. Comparisons of different baselines result.

	Dataset1					Dataset2
Model	Cov. Rate	Hits@1	Precision	Recall	F1	Cov. Rate	Hits@1	Precision	Recall	F1
HAN	-	-	0.7539	0.6917	0.6768	-	-	0.7203	0.6458	0.6246
XLNet	-	-	0.7514	0.7677	0.7349	-	-	0.7088	0.7149	0.6899
BERT-BiGRU	-	-	0.8560	0.8237	0.8281	-	-	0.8129	0.7651	0.7933
EEuEK	-	-	0.8417	0.8289	0.8307	-	-	0.7864	0.7895	0.7768
BGMA	-	-	0.8670	0.8319	0.8448	-	-	0.8103	0.7954	0.8113
DeBERTa-SSP	-	-	0.8656	0.8695	0.8640	-	-	0.8170	0.8310	0.8132
Pullnet	0.9188	0.8513	0.8769	0.8725	0.8653	0.9110	0.8375	0.8599	0.8429	0.8531
EmbedKGQA	0.9106	0.8590	0.8855	0.8639	0.8791	0.8846	0.8416	0.8721	0.8515	0.8511
RnG-KBQA	0.9338	0.8707	0.8755	0.8834	0.8808	0.9078	0.8505	0.8671	0.8584	0.8528
UniK-QA	0.9401	0.8757	0.8791	0.8815	0.8804	0.9102	0.8549	0.8699	0.8645	0.8603
SKP	0.9372	0.8669	0.8836	0.8785	0.8774	0.9076	0.8395	0.8650	0.8679	0.8596
no-sanding	0.9620	0.8664	0.9054	0.8933	0.8773	0.9349	0.8486	0.8607	0.8520	0.8535
Our	0.9620	0.8866	0.9350	0.9006	0.9132	0.9349	0.8552	0.8988	0.8741	0.8877

As can be seen from Table 3, our model achieves better experimental results in two datasets. The text classification based model is significantly less effective than the answer selection based model and KBQA-based model for event assignment, which is due to the fact that the text classification model is the worst because it only considers the relevant information of the event itself and cannot obtain more additional information. However, there is also a gap between the two text classification models, which is due to the gap in the effectiveness of the models for event text feature extraction, with the HAN model using BiGRU combined with the attention mechanism to extract text features, while XLNet is a language model trained on a large-scale corpus, thus having a better feature extraction capability and thus achieving a better result than HAN.

The effect of the answer selection based model is significantly lower than our model, because the answer selection based model only takes into account the text features of the “sanding” closely related to the department, and does not take into account the a priori knowledge of historical events, which illustrates the effectiveness of the knowledge graph based on historical events. Meanwhile, the BERT-BiGRU-based model, EEuEK and BGMA also have a gap, which is also due to the gap between the text extraction effect of different text feature extractors. Compared with BERT-BiGRU-based model, which uses BERT model as the feature extraction model of event text, EEuEK uses RoBERTa model, which is more effective, and thus achieves better results. BGMA simultaneously uses BERT and GCN as feature extraction models, thus achieving a better model effect than EEuEK. DeBERTa SSP uses the better pre trained model DeBERTa as the feature extraction model, while adding the Spans in Same Paragraph (SSP) strategy, thus achieving good model performance.

Compared with our model, the effects of Pullnet, EmbedKGQA, RnG-KBQA, UniK-QA and SKP are also significantly lower than it, because these models only consider the a priori knowledge of the historical event knowledge graph, but do not use the text features of the “sanding” closely related to the department. From the perspective of coverage rate, the subgraph retrieval effect of the baseline model is inferior to our model, so the model in this paper has achieved the optimal effect. From the results of “No-sanding” model, the effect of this model is only slightly lower than that of RnG-KBQA model, which can reach the same level as those most advanced KBQA models, which also fully illustrates the performance of the model in this paper. It is worth noting that the experimental results show that the effect of KBQA-based models is slightly higher than that of answer selection based models, which indicates that the a priori knowledge of the historical event is better than the “sanding” text features.

According to the comparison results between “No-sanding” model and our model, the effect of other evaluation indicators except coverage rate is lower than that of the model in this paper, which indicates that the retrieval embedding based on “sanding” text can effectively improve the assignment performance of the model. Since the subgraph retrieval process of the two models is the same, the coverage rate is the same. Since the “sanding” text is closely related to the department, the embedded “sanding” text can add the feature related to the department, so the retrieval and embedding based on the “sanding” text can effectively improve the performance of the model.

The horizontal comparison of the results of the models on the two data sets shows that with the increase of the number of assignment departments, the effects of all models decline to a certain extent. However, the effect of our model is still optimal, and the decrease of the model effect is also the smallest, which shows that our model certain advantages in dealing with the hotline assignment problem, and the stability and generalization performance of the model are relatively high.

Based on the above experimental results, our model achieves optimal results in both longitudinal baseline model comparisons and cross-sectional comparisons on different datasets. This is due to the fact that our model uses the knowledge graph based on historical events and also incorporates the retrieval embedding based on “sanding” text, and when using the knowledge graph, the subgraph retrieval is utilized to narrow down the candidate set of answers in order to improve the accuracy of the model’s prediction, and when using “sanding” text, the text retrieval algorithm is designed to obtain the most relevant text, thus further improving the model’s prediction accuracy. When using the “sanding” text, the text retrieval algorithm is designed to obtain the most relevant text to further improve the model performance.

Incomplete KG Analysis

The model in this paper needs to retrieve the path between the head entity and the answer entity in the knowledge graph to answer the question, and the answer is restricted to retrieve the entities in the subgraph. Therefore, in order to verify the effectiveness of our model in the case of incomplete KG, we perform random elimination of KG edges, and tested the assignment performance of our model under the condition of retaining 30%, 50%, and 80% edges of the knowledge graph.

Tables 4 and 5 show the main experimental results with different settings of incomplete KG, comparing Pullnet, EmbedKGQA, RnG-KBQA, UniK-QA, SKP and “No-sanding” models, we can see that the Hit@1 performance of RnG-KBQA model outperforms the other models regardless of the KG completeness setting when only KG information is used. However, our model outperforms the RnG-KBQA model in the case of incomplete KB due to the use of KG and the “sanding” text information, which indicates that the “sanding” text features closely related to the department can effectively improve the model performance.

Table 4. KG incomplete experimental results of Dataset1.

	30% KG		50% KG		80% KG
Model	Coverage rate	Hits@1	Coverage rate	Hits@1	Coverage rate	Hits@1
Pullnet	0.6537	0.5497	0.8172	0.7655	0.8867	0.8353
EmbedKGQA	0.6317	0.5871	0.8040	0.7729	0.8788	0.8338
RnG-KBQA	0.6627	0.6121	0.8269	0.7826	0.9026	0.8434
UniK-QA	0.6594	0.6164	0.8178	0.7887	0.9113	0.8487
SKP	0.6655	0.6002	0.8100	0.7633	0.9054	0.8388
No-sanding	0.7133	0.5993	0.8513	0.7796	0.9253	0.8416
Our	0.7133	0.6489	0.8513	0.7925	0.9253	0.8614

Table 5. KG incomplete experimental results of Dataset2.

	30% KG		50% KG		80% KG
Model	Coverage rate	Hits@1	Coverage rate	Hits@1	Coverage rate	Hits@1
Pullnet	0.6459	0.5359	0.8002	0.7359	0.8745	0.8093
EmbedKGQA	0.6143	0.5737	0.7916	0.7449	0.8528	0.8136
RnG-KBQA	0.6543	0.5871	0.7989	0.7530	0.8752	0.8248
UniK-QA	0.6546	0.5899	0.7980	0.7595	0.8788	0.8258
SKP	0.6549	0.5824	0.7908	0.7541	0.8760	0.8128
No-sanding	0.6729	0.5838	0.8110	0.7527	0.8844	0.8223
Our	0.6729	0.6080	0.8110	0.7620	0.8844	0.8291

As shown in Figures 5(a) and 6(a), from the perspective of coverage rate, the subgraph retrieval effect of our model is optimal for all kinds of incomplete KGs, so the optimal model effect is achieved. In addition, according to the results of the cross-sectional comparison between our model and other KBQA models on different settings of incomplete KGs, it can be seen that the indicators of the model are decreasing as the number of edge removal increases, which shows that the KG can effectively improve the model effect, and the more complete the KG model is, the better the effect is.

Figure 5. KG incomplete experimental comparison of Dataset1.

Figure 6. KG incomplete experimental comparison of Dataset2.

As shown in Figures 5(b) and 6(b), the performance of our model can be improved by using the “sanding” text features under different settings of incomplete KGs, and the effect on the model performance is increasing with the increase of KG completeness (except for 30% KB, which may be due to the effect of excessive edge connection removal). Based on general common sense, as the KG completeness increases, the impact of other types of features on the performance of the model becomes smaller and smaller, while the performance of our model is significantly enhanced by using “sanding” text with the condition of KG completeness increase. This may be due to the fact that the KG built in this paper is currently incomplete enough. However, from the experimental results, regardless of the KG completeness setting, the effect of our model is optimal, which fully illustrates the effectiveness of the model. At the same time, it can be proved that the performance of KBQA is effectively improved by using the “sanding” text method in the case of incomplete knowledge base.

Inference Time Analysis

The assignment of hotline events in the proposed model consists of two sequential processes: the construction process of a knowledge graph based on event extraction and the “event-department” matching process based on subgraph retrieval and text retrieval. The experimental results are presented in Table 6.

In the experimental environment, the average time taken by the knowledge graph construction process, based on event extraction, is approximately 3.8e-03 seconds, while the average time taken by the “event-department” matching process, based on subgraph retrieval and text retrieval, is about 6.0e-03 seconds. In the same experimental environment, the overall time for SKP to complete the inference and assignment task is 30.7e-03 seconds, while the time required by our model is 16.2e-03 seconds. It can be observed that the inference speed of the model in this paper is in milliseconds, In the case of similar performance, the proposed model can also have better efficiency than the compared baseline model on meeting the increasing demand for 12,345 hotline events assignment at a rapid pace.

Table 6. Comparison of time consumption of models.

Model	Inference Time（ms）	Hits@1
knowledge graph construction process	3.8	-
“event-department” matching process	6.0	-
SKP	30.7	0.8669
Our	16.2	0.8866

Conclusions

Due to the problem that the existing government hotline event assignment methods cannot effectively utilize a priori knowledge and only assign events based on the textual content of events, resulting in low assignment accuracy, we proposed a government hotline event assignment model based on KBQA method to effectively utilize the a priori knowledge of historical events and the a priori knowledge of “sanding” to improve the performance of event assignment. Firstly, we use the event extraction model to obtain the key information of events and construct the knowledge graph based on historical events. Secondly, using the event trigger, event type, and argument role obtained from the event extraction to retrieve the subgraph, and the fusion-GCN model is used to obtain the subgraph representation vectors. Then, text retrieval method is used to get the most relevant “sanding” text, and a BERT is used to obtain the “sanding” representation vector and another BERT is used to obtain the event text feature representation vector set. Finally, the event text feature representation vector set, the graph representation feature vector and the “sanding” representation vector are fused to predict the department score, and the optimal assignment department is output according to the score ranking results. The experimental results also prove the effectiveness of our model, and the KG incompleteness analysis also shows that the KG completeness of this paper needs to be further improved.

However, it should be noted that our model also has some limitations; for example, the amount of information contained in the constructed graph may have a large impact on the prediction results due to the relatively heavy reliance of our model on the completeness and accuracy of the constructed knowledge graph. In the future, we will consider completing missing event-related entities in the knowledge graph to further improve the KG completeness and thus improve the assignment performance of the model. Meanwhile, we can design more efficient graph algorithms and model structures to further improve the model’s inference speed. Currently, our model is mainly applied to the Chinese government hotline field; for broader application scenarios, we will conduct further research on English-related data in the future.

Acknowledgements

We would like to thank the anonymous reviewers for their valuable comments and suggestions that helped improve the quality of this manuscript.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.

Additional information

Funding

The work was supported by the Key Technologies Research and Development Program of Anhui Province [202104a05020071].