Semantic-aware event link reasoning over industrial knowledge graph embedding time series data

Abstract

The time series data in the manufacturing process reflects the sequential state of the manufacturing system, and the fusion of temporal features into the industrial knowledge graph will undoubtedly significantly improve the knowledge process efficiency of the manufacturing system. This paper proposes a semantic-aware event link reasoning over an industrial knowledge graph embedding time series data. Its knowledge graph skeleton is constructed through a specific manufacturing process. NLTK is used to transform technical documents into a structured industrial knowledge graph. We employ deep learning (DL)-based models to obtain semantic information related to product quality prediction using time series data collected from IoT devices. Then the prediction information is attached to the specified node in the knowledge graph. Thus, the knowledge graph will describe the dynamic semantic information of manufacturing contexts. Meanwhile, a dynamic event link reasoning model that uses graph embedding to aggregate manufacturing processes information is proposed. The implicit information with industrial temporal knowledge can be further mined and inferred. The case study has shown that the proposed knowledge graph link reasoning reflects dynamic temporal characteristics. Compared to the classical knowledge graph prediction models, our model is superior to the baseline methods.

KEYWORDS:

• Time series data
• data modelling
• semantic fusion
• industrial knowledge graph
• link reasoning

1. Introduction

With the booming rise of AIoT, AI-driven data science is enabling a rapid shift towards more intelligent manufacturing (Theorin et al. 2017). Currently, the number of IoT devices in the production workshop of the manufacturing industry is rising continuously. The data generated by information exchange between devices is rapidly expanding. The massive data includes static data, such as technical documents, and dynamic time series data generated by real-time perception of devices (Zheng et al. 2018, 2019; Wang et al. 2019). These data are characterised by structure and non-structure, time-varying, multi-source, and semantic relationship complexity (Lee et al. 2018; Sisinni et al. 2018). Semantic technology like ontology (Kadiri and Kiritsis 2015) and semantic web (Lu and Xu 2017) are essential for processing such data. However, the large amount of static information in the technical documentation is diverse.

Although it is possible to construct the semantic description related to device resources using ontology technology, it is difficult to form a unified, standardised, and fine-grained semantic description. To build large-scale fine-grained structured semantic knowledge, a knowledge graph (Zhu and Iglesias 2017) is introduced to build complex semantic information, which describes entities and relationships in the physical world as nodes and edges. At present, it has obtained preliminary application research in the vertical field (Noura et al. 2019; Zheng et al. 2021). Data technology represented by deep learning of statistical AI (artificial intelligence) has played a great role in data noise cleaning, feature compression, predictive analysis, and other processes (Ardolino et al. 2018; Fang et al. 2020; Tronvoll et al. 2020). But the contextual relationship between dynamic data and static data is poor. With the increasing complexity of the collaborative interaction process, there is a serious lack of effective data modelling methods to fuse the two types of data to form a unified semantic knowledge base. Moreover, the semantic reasoning dynamic events in manufacturing processes data also lack effective mining of implicit information to discover event causal information for driving intelligent production hardly. Using deep learning methods still faces huge challenges in dealing with the above issues. For example, the number of occurrences in dynamic data is often very small for some rare fault feature types, which seriously affects the prediction effect of using deep learning models. In this regard, some researchers introduced production knowledge (knowledge graph) as a guided model to predict and solve these issues (Zhou, Bao, Li, et al. 2021). These models can learn production knowledge and convert it to embedding for large-scale computing. However, only part of the information in the knowledge graph can be used. Moreover, the spatial topological dependence between time series data and device nodes cannot be fully obtained. The graph neural network of symbolic AI has the advantage of naturally handling Non-Euclidean structure data format. It can enable more interpretable manufacturing decision-making with augmented intelligence via integrating the merits of statistical learning with semantic knowledge and reasoning, which has seen significant attention (Kosasih and Brintrup 2021; Zhou, Bao, Chen, et al. 2021; Zhou, Hua, et al. 2021). The combination of statistical AI and symbolic AI technology is a feasible answer to solve the above challenges in industry.

Motivated by addressing the above-mentioned issues, we design a semantic-aware event link reasoning over an industrial knowledge graph embedding time series data. Firstly, collecting the time series data generated by multiple IoT device units and segment different time series data types. Modelling time series data is used as the input of the recurrent neural network model to predict product quality information. Then a temporal knowledge graph is built, which integrates dynamic time series semantic events and workshop knowledge from technical documents. We further propose a dynamic event link reasoning model to mine the deep implicit association information in the manufacturing stage. In the end, the experimental results demonstrate that our proposed solution can enhance the reasoning and mining ability of time series events significantly when compared to multiple existing solutions. To the best of our knowledge, this is the first work to address the transformation of industrial time series into semantic events for further use to implement knowledge link reasoning tasks in an industrial temporal knowledge graph.

In summary, the main contributions of this paper are as follows: (1) We make use of time-series data collected from IoT devices into a CNN-LSTM-based to generate events with temporal semantics. (2) We build an industrial temporal knowledge graph base by fusing dynamic time series semantic events and workshop knowledge from technical documents. (3) We propose a dynamic event link reasoning model to deal with evolutionary events with time series characteristics.

The remainder of the paper is structured as follows: Section 2 demonstrates related work. Section 3 details the unified model and framework of an industrial temporal knowledge graph. A case study is evaluated in Section 4. Finally, Section 5 concludes the paper and provides future insights.

2. Related work

Firstly, we introduced some recent research on RNN (recurrent neural network) for the industry’s prediction analysis of time series data. Secondly, we investigated recent research on static knowledge graphs in industry. At the same time, the processing of time series knowledge and the graph network related to this paper is also discussed.

2.1. Predictive analysis of dynamic time series data

Our research focuses on predicting and analyzing dynamic time series data, which is mainly related to deep learning techniques, such as RNN (Ho, Xie, and Goh 2002). The primary purpose is to provide a model framework. Anomaly detection and fault prediction are two of the important applications in the industry (Cook, Misirli, and Fan 2020), which employed implicit patterns from historical time series data collected in real-time at a device to predict device health status in the future.

(Lu et al. 2018) proposed a new deep architecture by combining LSTM (long short-term memory) with DNN (deep neural network) to solve the issue of early fault detection. At the same time, (Wang et al. 2020) designed a CNN(convolutional neural network) and DLSTM (deep long-short term memory) model. The method can be used to get a performance degradation assessment of rolling bearing using sensor time-series data. Based on the CNN-DLSTM, (Wang et al. 2021) further introduced transfer learning to achieve the fault diagnosis of rolling bearings under multiple working conditions.

Previous methods are mainly based on RNN to model, analyse, and predict the time series data generated by IoT devices. These methods ignore the semantic information dependence behind industrial time series. Until now, in the aspect of prediction, it is mostly the optimisation of the prediction model. However, for some rare fault feature types, due to the lack of semantic information related to fault features, the diagnosis and prediction effect based on the deep learning model will be greatly reduced. Different from previous methods, we consider combining the deep learning model with semantic information of knowledge graph in the production workshop. At the same time, we associate each type of device with its own technical knowledge and provide fault descriptions from the semantic level.

2.2. Static semantic data representation and embedding

Extensive studies have been done on semantic data representation and embedding. Ontology and knowledge graphs play an important role in semantic data representation (Lu and Asghar 2020). The ontology-based automatic design method proposed by (Fan et al. 2014) is applied to the intelligent rehabilitation system in the IoT, which can automatically design and reconfigure rehabilitation strategies and medical resources that can quickly respond to patient needs. (Liu et al. 2019) proposed a method based on DL to automatically associate multiple ontology rule bases with discovering new reasoning rules. They were then applied to the IoT with multi-domain knowledge.

However, with the increase of IoT devices, the complexity of semantic data from the technical documents related to devices also increases. Ontology is based on rule representation, which lacks flexibility and adaptability in fine-grained semantic descriptions of workshop and large-scale data processing efficiency. The knowledge graph is a structured semantic knowledge base, which has gradually been applied in the semantic collaboration of IoT (Li et al. 2020). Furthermore, to mining the implicit information from a knowledge graph, the triples in knowledge graph are usually transformed into corresponding low-dimensional vector embedding by using knowledge representation learning model (Bordes et al. 2013), graph convolution model (Kipf and Welling 2017), or language model (Vaswani et al. 2017).

The above knowledge representation is the processing and utilisation of static knowledge. However, dynamic knowledge nodes are not considered, so there is a lack of modelling for combined time series semantic data for knowledge graphs.

Recently, some researchers have tried to embed time series information into the relationships of knowledge graphs and deal with temporal knowledge graphs (García-Durán, Dumančić, and Niepert 2020). The effect of knowledge reasoning and link prediction is improved to a certain extent, but the granularity of time representation is coarse. On the other hand, there is a lack of time dependence. Re-NET (Jin et al. 2020) used a graph network to aggregate time information, which provides an idea for establishing the temporal dependency relationship.

At present, there is no prior research that models context-aware workshop knowledge and time series data with semantic information using graph embedding in the industry. In the face of massive dynamic and static data of IoT devices, there is little research on the unified utilisation of dynamic time series data and static resource data from the semantic level, guiding intelligent production. Related research (Li et al. 2020) focuses on using knowledge graphs to realise semantic cooperation among heterogeneous systems. This paper aims to explore a modelling method that can effectively associate and fuse dynamic time series data and static data between production resources and generate a unified industrial semantic knowledge base to drive implicit information mining and reasoning.

3. The unified model and framework of IoT semantic knowledge base

To study a modelling method that can effectively associate and fuse dynamic data and static data between production resources, it is necessary to generate a unified IoT semantic knowledge base to drive the method of implicit information mining analysis and causal reasoning of abnormal events. We proposed the unified model and framework of the temporal semantic knowledge base, as shown in Figure 1. The framework is mainly divided into three parts. Firstly, we introduced the time series data model based on CNN-LSTM-Attention originated from DL. Then, the knowledge graph of the workshop information from technical documents is constructed by NLP (natural language processing) tool. Then the feature information of time series data is fused with the constructed knowledge graph through semantic mapping. Finally, we aggregate the neighbour information of events by a graph neural network to learn and predict the specified events according to the industrial temporal knowledge graph formed by the fusion method.

Figure 1. The unified model and framework of industry semantic knowledge base.

(a) Time series data prediction model for obtaining semantic information; (b) The flow for building knowledge graph related to devices; (c) A dynamic event link reasoning model for mining the implicit information from the industrial temporal knowledge graph.

3.1. Semantic modelling of time series data

There are a variety of IoT device units in the workshop. Various IoT sensors generate multiple types of time series data during the production process. The integrated segmentation process of time series data of IoT perception is shown in Figure 2. This part of our solution is designed with the intention to obtain the predicted semantic information. Specifically, the time series data of device production units in the global IoT can be integrated and processed to effectively use time series data. Different device types and fault types will be detected. It is necessary to divide all the IoT perceptual time series feature data according to different prediction modules. The formed data conform to the specified device prediction event type, which is convenient for parallel transmission to the LSTM prediction model for specific event prediction, as shown in Figure 1(a). Therefore, we proposed a semantic model for time series data segmentation and prediction.

Figure 2. Integrated segmentation process of time series data of IoT perception.

The segmentation flow of time series data, showing time series data divided into different segments.

3.2. Definition 1: semantic modelling of time series data

According to the interval time T_h, the data acquisition sequence is divided into one node, which is defined as: (1) $X_{T_h}^i=(x_{T_h,1}^i,x_{T_h,2}^i,x_{T_h,3}^i,\dots ,x_{T_h,d}^i)i,h\in N$(1) $X_{T_h}^i\in R^d$ is the data value of the i-th data node at time T_h, which are d dimensions. The data of each dimension is represented by $x_{T_h,k}^i$($1\le k\le d$). The number of all nodes is represented by N. $X_T^i=\text{{ }{X}_{T_1}^1,X_{T_2}^2\text{,}{X}_{T_3}^3\text{,}\dots \text{,}{X}_{T_t}^i\}$ represents all node sequences corresponding to continuous time periods.

A sample label is represented as $Y^i=(Y_{T+1}^i,Y_{T+2}^i,Y_{T+3}^i,\dots ,Y_{T+\tau }^i)i\in N$. In which Yⁱ represents the target sequence data of the i-th data node with window length τ.

CNN-BiLSTM-Attention is used as the fault diagnosis model of time series data. It is specifically demonstrated as follows: (2) $f_t=\sigma \left({\mathrm{W}}_f{\text{X}}_{T_h}^i+{\mathrm{Q}}_f{h}_{t-\text{1}}\text{ + }{\text{b}}_f\right)$(2) (3) $i_t=\sigma \left({\mathrm{W}}_i{\text{X}}_{T_h}^i+{\mathrm{Q}}_i{h}_{t-\text{1}}\text{ + }{\text{b}}_i\right)$(3) (4) $o_i=\sigma \left({\mathrm{W}}_o{\text{X}}_{T_h}^i+{\mathrm{Q}}_o{h}_{t-\text{1}}\text{ + }{\text{b}}_o\right)$(4) (5) $c_t=f_t\ast c_{t-1}+i_t\ast \tanh \left({\mathrm{W}}_c{\text{X}}_{T_h}^i+{\mathrm{Q}}_c{h}_{t-\text{1}}\text{ + }{\text{b}}_c\right)$(5) (6) $h_t=o_t\ast \tanh \left(c_t\right)$(6)

The output state information is represented as h_t. To bring different diagnostic effects to different time series features, attention mechanism is introduced. The ${\alpha }_t^k$, which is the attention weight of the k^th time series features, is represented as follows. (7) $\begin{array}{rl}& \begin{array}{l}{\alpha }_t^k={\displaystyle \frac{exp\left(e_t^k\right)}{{\sum }_{i-1}^n\exp \left(e_t^i\right)}}\\ e_t^k=\tanh \left(W_e{h}_t+{\text{b}}_e\right)\end{array}\end{array}$(7) (8) $\begin{array}{rl}& p_t=\sum _{t=1}^T{\alpha }_t^k{h}_t\end{array}$(8) Where $e_t^k$ calculates the time series features classification of h_t, corresponding to the LSTM output h_t, thereby obtaining a new output characteristic value p_t containing attention information. To realise fault prediction, normalise the time series information based on softmax function, and transform it into probability operation to predict the fault types, it is described as shown in Equation (9): (9) $\text{p}\left(y|{\text{X}}_{T_h}^i\right)=softmax\left(s\left(p_t\right)\right)$(9) The overlap ratio of input time series data is set to 50% in every training process to capture the information between time series. Finally, product quality prediction features are obtained. Based on a Direct Mapping of Relational Data to RDF (D2R)¹ and database to RDF-Query(D2RQ),² fault types and corresponding time nodes are linked to corresponding devices to form semantic information of diagnostic features.

Another kind of data type with temporal characteristics is the image, which is an unstructured data form. Products need to be checked for defects at some point in manufacturing. There are various types of defect features of products in the industry, which are often captured as data in the form of images. This, in turn, enables image defect detection using DL techniques. Moreover, the detected images are time-stamped to facilitate quality traceability. Therefore, this kind of data has temporal properties. Defect detection in industrial images can also be achieved using a series of DL methods. Fortunately, our team has already conducted research on defect detection in welding images using CNN and LSTM techniques and has achieved significant success (Liu et al. 2020, 2021). Thus, this method will be used for image detection in this paper. The contribution of defect detection using DL is to get the semantic information for supporting the knowledge link reasoning in Section 4.3.

3.3. Time series data fusion with knowledge graph

In this section, we use a knowledge graph to capture the dependency relationship of the device’s spatial position. Besides, we associate the semantic of time series diagnosis with device knowledge graph through semantic mapping.

1. Modelling of static knowledge graph of workshop resources

The static knowledge graph from workshop resources consists of a series of triples. It mainly integrates the semantics relationships between devices, devices and workpieces, and devices and technical documents. It also integrates the attribute information of devices, workpieces, and tools. We give the data processing flow to build a knowledge graph as Figure 3.

Figure 3. The data processing flow of the NLTK and D2R.

The data processing flow for building industrial knowledge graph, showing the upper section is using NLTK to process unstructured data, and the lower section is using D2R to process structured data.

Concretely, we extract the structured and non-unstructured data related to the workshop devices from the databases behind the enterprise information management platforms, such as MES, ERP. Our intention is to represent the acquired data in the form of triples as (e_s,r,e_o). e_s represents the subject entity. r represents relationships. e_o represents the object entity.

We use Natural Language Tool Kit (NLTK)³ v. 3.5 to transform unstructured data, typically technical documents, into structured information to form triples. As NLTK does not support Chinese word segmentation, firstly, we use jieba,⁴ a segmentation tool, to complete Chinese text segmentation for compatibility with NLTK. Next, part of speech tagging is performed to identify named entities among the words. Finally, we use relationship recognition to search for possible relationships between similar entities. This is usually done by finding all triples as (e_s,r,e_o). The static knowledge graph of workshop devices is established, as shown in Figure 1(b). In the static knowledge graph, for structured data, we use the D2R technique to map tables, columns, rows and cells in the database directly to their corresponding classes, properties, instances/resources and literals respectively. Based on the inclusion and hierarchical relationships defined in Table 1, devices’ static knowledge and semantic information are organised. The attribute value is used as a node. The attribute is used as an edge. The attribute is extracted by the D2R. The attributes mainly contain the specific description of the entity, such as the factory parameters, type, and number of the device, which can be extracted through the device description documents.

1. Link between knowledge graphs and temporal events

Table 1. Description of semantic relationship.

Type	Name	Definition	Description
Inclusion relationship	Perform a weld	Inclusion relationship between device and workpiece	$◯⟹performaweld◯$
	Make an image detection	Inclusion relationship between DL-based model and workpiece image	$◯⟹\text{make an image detection}◯$
	Detect a feature	Inclusion relationship between workpiece image and detection result	$◯⟹\text{detect a feature}◯$
Hierarchical relationship	Sequence	Subordinate relationship of processing between device and device	$◯⟹Sequence◯$
	Parallel	Parallel relationship between device and device	$◯⟹Parallel◯$

The original data collected on the device has a timestamp in the default state, which means a set of time series data are generated at a specific time for a specific device. They represent the processing status and processing capability of the device in a specific time period. Therefore, it is necessary to build the relationship model between the state characteristics of the time series for devices, so as to obtain the knowledge of the device with time characteristics. The Section 3.1 has used RNN to predict the diagnostic feature information based on the device’s time series data, and built a static knowledge network of devices. In this part, we consider unifying the semantic information of time series diagnosis features with the knowledge graph related to devices. First, the paper regards sequence diagnosis at each time as a temporal event, which has a time label and attributes values of corresponding characteristics. Then, the characteristic value of each temporal event is regarded as the knowledge with time series, and connected to the static knowledge graph of the devices to form the temporal knowledge graph of the devices (TKG), which can be expressed as follows: $\text{TKG}=<E,R,S,T>$ Where E is a set of knowledge nodes. R is a set of relational edges. S is the set of all static triples. T is the set of all connected quadruples of temporal events. The quadruple T_i can be represented as T_i = (e_s,r,e_o,t_i)|T_i∈T. t_i represents the timestamp for a unit time. It can also be converted into a form of triple, T_i = (e_s, $r_{t_i}$,e_o)|T_i∈T, which increases the semantic relationship between entities without changing the existing knowledge elements by introducing a timestamp on the relationship. Based on the TKG, it is possible to realise the fusion and organisation of time series data knowledge of workshop devices, and provide a base of time series semantic knowledge for fault prediction of subsequent devices.

We give the unified semantic link algorithm for knowledge of temporal events in algorithm 1.

Table

Algorithm 1: the unified semantic link algorithm between time series and knowledge graph
Input:  Array(E) is the set of events and attributes, Array(D) is the set of static device knowledge nodes.
Output:  DE is the set of quadruples with event knowledge, DK is the set of dynamic knowledge nodes.
1:	DK ←ϕ
2:	DE = Array(D)
3:	Ti = [t1, t2, t3, … ,ti, … , tn]// Ti is the time when event i occurred
4:	Rij = [Ri1,Ri2, Ri3, … ,Rij, … , Rin]
5:	foreach Ei in Array(E) // Traverse all events, Ei is the i-th event
6:	add Ei to DE
7:	foreach Aij in Ei // Aij is the j-th feature of the i-th event
8:	if Aij in DE
9:	Rij = [Ri1,Ri2, Ri3, … ,Rij, … , Rin]//Rij represents the relationship between Ei and Aij
10:	new Kt = (Ei,Rij,Aij,Ti) // Kt represents the quadruple knowledge with time series
11:	add Kt to DK
12:	end if
13:	else
14:	add Aij to DE   if Ai does not exist, add it to the knowledge base to complete the knowledge
15:	Rij = [Ri1,Ri2, Ri3, … ,Rij, … , Rin]
16:	new Kt = (Ei,Rij,Aij,Ti)
17:	add Kt to DK
18:	end if
19:	end foreach
20:	end foreach
21:	return DE, DK

3.4. Temporal knowledge graph link reasoning

To further use and mine the generated temporal knowledge graph to predict implicit information between devices, we employ graph convolutional network (GCN) to aggregate neighbourhoods information. It can generate node-level embeddings by extracting aggregated information about neighbour nodes in the graph network to support downstream tasks. R-GCN (Schlichtkrull et al. 2018) has a good ability to express multi-relational directed graphs, which is suitable for processing knowledge graphs. CompGCN (Vashishth et al. 2020) can use the knowledge graph composition operator to represent multi-relational directed graphs, which are more generalised. Therefore, considering using GCN model to aggregate time series related first-order neighbourhood information.

However, in this research, the aggregated information of the graph structure not only contains the time series information, but also contains the relevant information of the corresponding event neighbourhood information. Nevertheless, the GCN model alone does not effectively handle temporal event information. The knowledge graph fused with dynamic and static data can be regarded as a combination of events with time series. RNNs are well-suited to learning information with time-series characteristics. It will provide more relevant information to mine the dependencies relationship between event sequences, and make greater utilisation of dynamic temporal knowledge graph information.

Inspired by the benefits of the two methods described above, we propose a strategy TKEGN (temporal knowledge evolutionary graph network) by combining the CompGCN and Transformer (Vaswani et al. 2017) to deal with events with time series characteristics. Because the fused workshop data has good series properties in the time direction and has a limited length, it allows us to learn temporal events based on the Transformer model. Also compared with RNN, the Transformer model can process data in parallel, improving the calculation accuracy. It is worth noting that the frameworks we propose can accommodate multiple languages. For computers to perform reasoning operations, it is simply a matter of linguistic coding whether they can handle different languages efficiently.

Concretely, assuming that a node is the central device, the CompGCN model can obtain the topological relationship between the device and the surrounding devices. The graph network of the device is semantically linked with the time series feature attributes at different times. That is, it shows that the multi-relational partial subgraph is related to a certain device in Figure 1(c). According to the aggregation information at different times, the entity vector and the relationship vector at different times are formed.

Therefore, the feature of predicting device fault state at different time can be obtained. Furthermore, the embedded vector of the aggregate information of the dependence relationships between the feature and the device is obtained.

The aggregate function is expressed as follows: (10) $\begin{array}{rl}\mathcal{G}\left(E_t\right)& =h_{e_{o}t}^{k+1}=f\left(\sum _{(e_s,r)\in \mathcal{N}\left(e_{o}t\right)}{\mathit{W}}_{\lambda \left(r\right)}^k\varphi \left(h_{e_s}^k,h_r^k\right)\right)\end{array}$(10) (11) $\begin{array}{rl}{h}_r^{k+1}& ={\mathit{W}}_{\text{rel}}^k{h}_r^k\end{array}$(11) $h_r^{k+1}$ is the representation of a relationship r after k layers.

Specifically, the vector z = {z₁, z₂,  … , z_n} generated by the aggregation of CompGCN is used as the input of the Transformer to predict the next moment based on the state information of the previous moments. To this end, we give the definition of events with time series characteristics.

The event E_t of the head entity e_s at time t is related to the head entity e_s in the event E_t-K:t−1 corresponding to the first k times. The r_t at time t depends on the event $E_{t-k:t-1}^{\left(e_s\right)}$ corresponding to the head entity e_s. At the corresponding moment t, e_ot depends on the head entity e_s and the event $E_{t-k:t-1}^{\left(e_s\right)}$ corresponding to the relationship r_t. Therefore, the probability of the time series triple aggregated at a time $\mathcal{G}\left(E_t\right)$ is expressed as follows: (12) $\begin{array}{rl}\text{P}\left({e_s}_t,r_t,{e_o}_t|\mathcal{G}\left(E_t\right)\right)& =\text{P}\left({e_o}_t|e_s,r,E_{t-k:t-1}^{\left(e_s\right)}\right)\\ & \cdot \text{P}\left(r_t|e_s,E_{t-k:t-1}^{\left(e_s\right)}\right)\\ & \cdot \text{P}\left({e_s}_t|E_{t-k:t-1}\right)\end{array}$(12) Among them, the probability of entity and relationship at time t is converted into an information entropy solution. [e_s:r:h_t−1(e_s,r)] represents the local information of all neighbourhoods, including e_s and r at the first times, which is: (13) $\begin{array}{rl}\text{P}\left({e_o}_t|e_s,r,E_{t-k:t-1}^{\left(e_s\right)}\right)& =softmax\left(\left[e_{\text{s}}:r:h_{t-1}(e_{\text{s}},r)\right]\right)\end{array}$(13) (14) $\begin{array}{rl}\text{P}\left(r_t|e_s,E_{t-k:t-1}^{\left(e_s\right)}\right)& =softmax\left(\left[e_{\text{s}}:h_{t-1}\left(e_{\text{s}}\right)\right]\right)\end{array}$(14) (15) $\begin{array}{rl}\text{P}\left({e_s}_t|E_{t-k:t-1}\right)& =softmax\left(\left[h_{t-1}\left(e_{\text{s}}\right)\right]\right)\end{array}$(15)

Using the Transformer model to update the time series triples (e_s,r_t,e_ot) at time t: (16) $\begin{array}{rl}& h_t(e_{\text{s}},\text{r})=Transforme{r}^1\left(\mathcal{G}\left(E_t^{\left(e_{\text{s}}\right)}\right),H_t,h_{t-1}(e_{\text{s}},\text{r})\right)\end{array}$(16) (17) $\begin{array}{rl}& h_t\left(e_{\text{s}}\right)=Transforme{r}^2\left(\mathcal{G}\left(E_t^{\left(e_{\text{s}}\right)}\right),H_t,h_{t-1}\left(e_{\text{s}}\right)\right)\end{array}$(17) (18) $\begin{array}{rl}& H_t=Transforme{r}^3\left(\mathcal{G}\left(E_t\right),H_{t-1}\right)\end{array}$(18) Where h_t−1(e_s) models the local information of all neighbourhoods of e_s in the first k moments. H_t−1 and H_t represent the global information in the first k moments and the current time series knowledge graph.

Based on the triples of historical k temporal events, predicting the future relationship between entities is essentially a multi-classification task, so the cross-entropy loss is used as the objective function: (19) $\begin{array}{rl}\mathcal{L}& =-\sum _{(e_s,r,e_o,t)\in G}(\log (\text{P}\left({e_o}_t|{e_s}_t,r_t\right)+{\lambda }_1\log \left(\text{P}\left(r_t|{e_s}_t\right)\right)\\ & +{\lambda }_2\log \left(\text{P}\left({e_s}_t\right)\right))\end{array}$(19) Where λ₁, λ₂ are hyperparameters.

The algorithm flow for mining the implicit information of events with time series characteristics is as follows:

Table

Algorithm 2: Reasoning algorithm for implicit information in time series data fusion with knowledge graph
Input: The triples for time series fusion with the knowledge graph Ti = (es,rt,eo)|Ti∈T, The local graph sequence{g1, g2, … , gt−1}, Number of sample N.
Output: Conditional distribution P((es,rt,eo)|g(Et+Δt)|g(E:t)), estimation triples Ti+Δt = (es+Δt,rt+Δt,eo+Δt)
1:	foreach {g1, g2, … , gt−1}←Ti  do
2:	Aggregate information g(Et) by Equation (10), Equation (11)
3:	Aggregated vectors z = {z1, z2, … , zn}
4:	end foreach
5:	t = {t1, t2, … , tn}∈ti
6:	foreach tm < ti+Δt, (m < i) do
7:	Compute the triples under aggregation $\mathcal{G}\left(E_t\right)$ at time tm  P((es,rt,eo)|g(Et+Δt)|g(E:t)) by Equation (12)
8:	Upgrade the triples based on Transformer by Equation (16), Equation (17), Equation (18)
9:	tm = tm+1
10:	return Ti+Δt = (es+Δt,rt+Δt,eo+Δt)
11:	end foreach

4. Case study

4.1. Types of industrial data

This case study is based on the specific processing tasks for welding thin-walled shell workpieces in an aerospace enterprise. There are three types of industrial data, as shown in Figure 4. A more complete industrial knowledge can be obtained by fusing three types of information, thus supporting the temporal knowledge reasoning model to achieve a top-level view to analyse the causes of faults.

1. Industrial welding time series data

Figure 4. Information on industrial data types and corresponding statistics.

(a) A preview example of the welding time series data and its data types; (b) Types of defects for welding images and their statistics; (c) An example of temporal knowledge graph for welding data and the statistics in the graph.

The first type is the time series data of the welding process of thin-walled parts collected by sensors on the welding equipment. It contains ten types of features that provide feature input for subsequent weld quality prediction models. The feature information is shown in Figure 4(a).

1. Industrial welding images data

The second type is the image data of the welding process collected by X-ray. After the shell has been welded, the object needs to be detected for defects. We summarise six common types of defects in shell welding with industrial practice, including burn-through, not welded through, welding air hole, welding bias, welding crack, and welding misalignment. The distribution and division of the data is shown in Figure 4(b). The collection of welding defect features by means of images, followed by the use of DL technology to identify the defect features of welding images, will improve the efficiency of advancing production.

1. Industrial welding temporal data

The third type is unstructured data from the welding technical manuals. They contain a wealth of information to guide the analysis and treatment of product machining and quality issues. we use NLTK to handle the technical manuals and build a welding knowledge graph dataset. Moreover, considering that each dynamic temporal event of the welding time series data and defect identification contain a corresponding timestamp, it can be equivalently understood as a time series event corresponding to a diagnosis result in this article. Therefore, the welding temporal knowledge dataset is built as shown in Figure 4(c). We collected the industrial weld data from 6/10/2021 to 7/10/2021. The dataset has temporally associated fact (e_s,r,e_o,t_i). So the dataset can be converted to {(e_s,r,e_o,t₁), (e_s,r,e_o,t₁₊1_t), … , (e_s,r,e_o,t_n)}. 1_t indicates a unit time granularity segmented by minutes to ensure each fact has a sequence of events. The temporal representation of the dataset is fine-grained. N_train, N_valid, N_test indicate the set of quadruples for temporal events, respectively. N_ent indicates the entities collected mainly from the industrial technical manuals. N_rel indicates the relationships types between entities.

4.2. Defect prediction for time-series data

4.2.1. Welding temporal data

A total of 1200 weld seam-related data from 30 days were collected and collated for training. Each weld contains 10 features, and each feature contains 240 data samples. 840 weld samples were used as the training set and the rest as the test set.

4.2.2. Prediction model parameter settings and evaluation metrics

In this paper, we establish a CNN-LSTM based prediction model. The interdependencies between different time series information are extracted using CNN, and then the before-and-after relationship of time series is extracted using LSTM. This allows for weld width prediction and image recognition.

1. Prediction model parameter settings

Firstly, the hyperparameters in the model and the debugging range of hyperparameters are clarified. Then the grid search method is used to traverse the values of all hyperparameters of the model. Finally, the optimal combination of hyperparameters is selected. The setting hyperparameters of the electron beam welding prediction model are summarised follows. The convolution layer filter size is 6. The convolution kernel size is 1. The number of LSTM neurons is 6. The number of LSTM network layers is 1. The number of neurons in the FC layer is 6. The learning rate is 1E-3. The batch size is 100.

1. Based methods and evaluation metrics

To validate the performance of the CNN-LSTM based weld prediction model and to explore the performance differences between the temporal and non-temporal methods, it is compared with three other models.

• LSTM model. The LSTM model alone is used with the same model parameter settings as the method proposed in this paper.

• CNN model. The CNN model is used alone, and the model parameters are set in the same way as the method proposed in this paper.

• BP neural network. The BP neural network model uses the four features of the current moment as input and the predicted weld width of the next moment as output.

The experiments in this paper judge the merits of model prediction accuracy and performance by three evaluation metrics: loss function, prediction bias, and root mean square error (RMSE). The calculation methods of these indicators are as follows. (20) $\begin{array}{rl}loss& =\frac{{\sum }_{i=1}^n|x_i^o-x_i^p|}{n}\end{array}$(20) (21) $\begin{array}{rl}deviation\left(x\right)& =\frac{1}{n}\left(\frac{{\sum }_{i=1}^n|x_i^o-x_i^p|}{x_i}\right)\end{array}$(21) (22) $\begin{array}{rl}RMSE& =\sqrt{\frac{1}{n}\sum _{i=1}^n{(x_i^o-x_i^p)}^2}\end{array}$(22)

Where $x_i^o$ and $x_i^p$ denote the real data and the predicted results, respectively. Smaller values of the above indicators indicate better prediction results.

In the training process, we record once in each training cycle. The smaller and more stable value of Loss convergence in the training process indicates that the model fitting error is smaller and more excellent. In the testing process, we record the results of each test separately, and the higher the accuracy of the model prediction indicates that its generalisation ability is stronger and has stronger applicability. Meanwhile, the smaller the RMSE indicates that the model prediction is more accurate.

4.2.3. Experimental results

Since the BP neural network is fundamentally different from the other three model methods in this paper, the loss function curves of the BP network are not comparable to the other three models in the paper. Figure 5 shows the variation curves of the loss function of the three models during the model training. The prediction results for 100 weld seam data are summarised in Table 2. According to the training Loss curves, the model’s Loss value levelled off within 40 training cycles, while the other two models still oscillated gradually.

Figure 5. Loss variation curves for the four models.

Four curves plotting the training loss of four models for predicting the quality of shell weld seam. Significantly higher prediction accuracy occurs in the green line.

Table 2. Performance comparison of the models’ predictions.

Model	RMSE	Accuracy
CNN-LSTM model	5.78	90.16%
CNN model	9.18	83.12%
LSTM model	11.22	77.13%
BP model	34.58	60.53%

The CNN-LSTM model used in this paper is found to have higher prediction accuracy through experimental comparison, indicating that the model can better extract the implied features between timing information. Based on the CNN-LSTM model can meet the electron beam welding prediction needs.

Moreover, the defect features of shell welding are identified using our team’s proposed method for defect detection in industrial welding images (Liu et al. 2020, 2021). The test results demonstrated that our team’s work is able to meet the weld defect feature identification of shells. Defect information will be mapped into a static knowledge graph network to record defect detection events, providing product quality traceability and analysis support.

4.3. Knowledge fusion and link reasoning

4.3.1. Welding knowledge graph with fused temporal features

Knowledge graph network of local temporal events welding thin-walled shell workpiece, as shown in Figure 6. This network contains information on the predicted weld width events of the workpiece welding process for a certain time period and the event information on the defect features identified by the images.

Figure 6. Knowledge graph network of local temporal events for welding thin-walled shell workpiece.

Industrial welding temporal knowledge graph network (local), with red nodes and yellow edges, showing the degree of association between nodes and edges.

Further, the temporal event information of Figure 6 is fused with the static knowledge of engineering technology. Thin-walled shell machining knowledge graph with fusion its welding temporal features are generated, as shown in Figure 7. The machining knowledge graph is considered to be global, including welding temporal information, defect features information, and the information from machining technology manuals. It is the integration of all related knowledge in the machining shop. In contrast, the experience of engineers is often only clear about the processing specifications of their own positions. There is little experience in accumulating knowledge involving cross-workshop or cross-processing chains. Therefore, knowledge-based reasoning from the TKEGN can considerably help engineers strengthen their skills in the cause-and-effect analysis of machining quality issues.

Figure 7. Thin-walled shell machining knowledge graph with fusion its welding temporal features.

Industrial welding temporal knowledge graph network fused static knowledge, with red nodes and edges, showing the association of temporal semantic characteristics and static knowledge between nodes and edges.

4.3.2. Knowledge reasoning baseline methods

Temporal knowledge link reasoning aims at discovering implicit semantic relationships between entities in a complex knowledge base. In other words, the semantic information in the knowledge base contains temporal properties. We mainly use the following methods⁵ as baseline methods to compare with the method mentioned in this paper.

TransE (Bordes et al. 2013): It represents the entities and relationships in the triples in the low-dimensional dense vector space and then calculates and infers triples.

DistMult (Yang et al. 2015): It represents the semantics of each entity with a vector and represents each relationship with a matrix to learn the reasoning.

ComplEx (Trouillon et al. 2016): It embeds entities and relationships into complex valued space to represent their embeddings, which can better reason the triple of asymmetric relationships.

ConvE (Dettmers et al. 2018): This method uses 2D convolutional neural networks to learn the triples embedding for knowledge graph reasoning.

R-GCN (Schlichtkrull et al. 2018): GCN establishes the encoder model of edge type and direction in a graph. A decoder is utilised to calculate link prediction for the knowledge graph.

CompGCN (Vashishth et al. 2020): Considering the type and direction of edges in the graph, it maps the relational embedded representation to the entity embedded space and combines with various decoders to calculate link prediction for knowledge graph.

TTransE, TA-TransE (Leblay and Chekol 2018): They are based on the TransE model and introduce a time processing strategy to adapt to the reasoning task of a temporal knowledge graph.

TA-DisMult (García-Durán, Dumančić, and Niepert 2020): Based on the DistMult model, it performs the reasoning task of the temporal knowledge graph.

ATiSE (Xu et al. 2020): It employs additive time series decomposition to incorporate time information into tiples representations, which performs the reasoning task of a knowledge graph.

In Table 3, we summarise the comparison of our model with several baseline models in terms of score function and space complexity. n_e, n_r, n_t, and n_token are numbers of entities, relations, time steps and temporal tokens. d is the dimensionality of embeddings. K denotes the number of layers in the models. β represents the base of the models. [r_p;t_seq] is the concatenation of the relation embedding and the sequence of temporal tokens. The number of |$\mathcal{R}$| indicates the total number of relationships in a knowledge graph.

Table 3. Comparison of our models with several baseline models for space complexity.

Models	Scoring function	Space complexity
TransE	$\Vert e_s+r_p-e_o\Vert$	$\mathcal{O}\left(n_{e}d+n_{r}d\right)$
DistMult	$<e_s,r_p,e_o>$	$\mathcal{O}\left(n_{e}d+n_{r}d\right)$
ComplEx	$\mathrm{R}\mathrm{E}(<e_s,r_p,{\overline{e}}_o>)$	$\mathcal{O}\left(n_{e}d+n_{r}d\right)$
ConvE	$f\left(\mathrm{v}\mathrm{e}\mathrm{c}\right(f\left(\right[\overline{{\mathbf{e}}_s};\overline{{\mathbf{r}}_r}]\ast \omega )\left)\mathbf{W}\right){\mathbf{e}}_o$	$\mathcal{O}\left(n_{e}d+n_{r}d\right)$
RotatE	$\Vert e_s\circ r_p-e_o\Vert$	$\mathcal{O}\left(n_{e}d+n_{r}d\right)$
R-GCN	$\Vert e_s+r_p-e_o\Vert$	$\mathcal{O}\left(\mathcal{B}K{d}^2+\mathcal{B}K\left|\mathcal{R}\right|\right)$
CompGCN	$\Vert e_s+r_p-e_o\Vert$	$\mathcal{O}\left(K{d}^2+\mathcal{B}d+\mathcal{B}\left|\mathcal{R}\right|\right)$
TTransE	$\Vert e_s+r_p+w_t-e_o\Vert$	$\mathcal{O}\left(n_{e}d+n_{r}d+n_{t}d\right)$
TA-TransE	$\Vert e_s+\mathrm{L}\mathrm{S}\mathrm{T}\mathrm{M}\left(\left[r_p;t_{seq}\right]\right)-e_o\Vert$	$\mathcal{O}\left(n_{e}d+n_{r}d+n_{token}d\right)$
TA-DistMult	$<e_s,\mathrm{L}\mathrm{S}\mathrm{T}\mathrm{M}\left(\left[r_p;t_{seq}\right]\right),e_o>$	$\mathcal{O}\left(n_{e}d+n_{r}d+n_{token}d\right)$
ATiSE	${\mathcal{D}}_{\mathcal{K}\mathcal{L}}\left({\mathcal{P}}_{e,t},{\mathcal{P}}_{r,t}\right)|$	$\mathcal{O}\left(n_{e}d+n_{r}d\right)$
TKEGN (Ours)	$<e_s,\text{Transformer}\left(\left[r_p;t_{seq}\right]\right),e_o>$	$\mathcal{O}(K{d}^2+\mathcal{B}d+n_{token}d$
		$+\mathcal{B}\left|\mathcal{R}\right|)$

Overall, our model is the most comprehensive, encoding relationship and direction information more generally. At the same time, the information of temporal features is more effectively characterised.

4.3.3. Evaluation metrics

The performance evaluation of semantics-aware fusion of time series data with the knowledge graph proposed in this paper is mainly considered from three aspects.

We use the first hit rate indicator (hit@1), the top 3 hit rate indicator (hit@3), the top 10 hit rate indicator (hit@10), and the mean reciprocal rank indicator (MRR) to evaluate the comprehensive ability of semantics-aware fusion of time series data with knowledge graph proposed in this paper (Bordes et al. 2013).

Specifically, it is necessary to predict the tail entity e_o of a link (e_s,r,?,t) or the head entity e_s of a link (?,r,e_o,t) through the existing quaternion of the temporal knowledge graph.

In detail, they are defined as follows: (23) $\begin{array}{rl}hit@1& =\frac{1}{\left|N\right|}\sum _{(e_s,r,e_o,t)\in N}\mathrm{i}\mathrm{n}\mathrm{d}\left(\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}\right(e_s,r,e_o,t)\le 1)\end{array}$(23) (24) $\begin{array}{rl}hit@3& =\frac{1}{\left|N\right|}\sum _{(e_s,r,e_o,t)\in N}\mathrm{i}\mathrm{n}\mathrm{d}\left(\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}\right(e_s,r,e_o,t)\le 3)\end{array}$(24) (25) $\begin{array}{rl}hit@10& =\frac{1}{\left|N\right|}\sum _{(e_s,r,e_o,t)\in N}\mathrm{i}\mathrm{n}\mathrm{d}\left(\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}\right(e_s,r,e_o,t)\le 10)\end{array}$(25) (26) $\begin{array}{rl}MRR& =\frac{1}{|N|}\sum _{(e_s,r,e_o,t)\in N}\frac{1}{\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}\left(e_s,r,e_o,t\right)}\end{array}$(26)

In these expressions, the symbol N represents the set of all test samples and the symbol |N| represents the number of quaternions in the set. ind(·) is an indicator function and used to check whether the triple (e_s,r,e_o,t) is ranked in the target item corresponding to the interference list.

First, the trained model is used to calculate the corresponding score for each time series quadruple in the interference list, and the final prediction ranking of the time series quadruple is obtained through the relationship prediction method, as shown in Equation (27). (27) $\begin{array}{r}rank\left(e_s,r_i,e_o,t\right)=\frac{rank\left(?,r_i,e_o,t\right)+rank\left(e_s,r_i,?,t\right)}{2}\end{array}$(27) rank(?,r,e_o,t), rank(e_s,r,?,t) represent the basic ranking results of the head entity and tail entity corresponding to the test set quadruple in the relationship prediction, respectively.

4.3.4. Experimental results

To analyse the performance of temporal knowledge link reasoning, we separately select a few specific relationships (‘perform a weld’ and ‘make an image detection’) and generate relationships subsets of the dataset. The proposed method is evaluated based on the welding dataset in Figure 4(c). Performance comparisons of the reasoning models on relationship ‘perform a weld’ and ‘make an image detection’ of the welding dataset are shown in Figures 8 and 9, respectively.

Figure 8. Performance comparisons of the reasoning models on relationship ‘perform a weld’.

A four section bar graph in twelve models plotting the accuracy of reasoning ability of four evaluation metrics, showing the performance of different reasoning models on relationship ‘perform a weld’.

Figure 9. Performance comparisons of the reasoning models on relationship ‘make an image detection’.

A four section bar graph in twelve models plotting the accuracy of reasoning ability of four evaluation metrics, showing the performance of different reasoning models on relationship ‘make an image detection’.

We can see that the validation results of our model can enhance the performance comparisons of the temporal knowledge link reasoning in specific relationships. This means that temporal information plays a crucial role in the knowledge links. Because the static models are unable to use temporal information in knowledge link reasoning, they usually perform worse than our models.

Furthermore, taking into account the evaluation of the comprehensive reasoning ability of the proposed method, we have selected one of the models with and without temporal characteristics to compare with our method for comprehensive performance. We give the calculation results of the MRR over time in the welding dataset, as shown in Figure 10. The dataset has a fine-grained granularity in the time segment. The results show that the proposed method also has a good generalisation ability for future temporal knowledge prediction.

Figure 10. Performance of the comprehensive reasoning ability over time in the welding datasets.

A three line graph plotting the Mean Reciprocal Rank of three models for the comprehensive reasoning ability. Significantly higher reasoning accuracy occurs in the green line.

Table 4 gives a comparison of the performance of our model with other baseline methods. Some insights can be obtained. The proposed method can learn the implicit information in the knowledge graph fused with time series information, and its prediction performance is better than the existing baseline methods.

Table 4. Comparison of prediction accuracy between the temporal models and the static models in welding temporal knowledge graph.

		Welding temporal knowledge graph
	Methods	MRR	H@10	H@3	H@1
Static models	TransE	.283	.353	.279	.194
	DistMult	.337	.438	.312	.238
	ComplEx	.341	.466	.390	.312
	ConvE	.480	.549	.458	.331
	RotatE	.362	.474	.353	.285
	R-GCN	.357	.483	.344	.273
	CompGCN	.498	.592	.476	.415
Temporal models	TTransE	.265	.334	.203	.177
	TA-TransE	.272	.360	.267	.186
	TA-DisMult	.404	.562	.430	.317
	ATiSE	.518	.656	.542	.446
	Proposed Method	.549	.673	.566	.482

To further illustrate the link reasoning for the temporal knowledge found by our model, we give an example as shown in Table 5. Taking the second edge ‘Make an image detection, not specified below: 2021-07-04’ as an example, Our model reasoning about the knowledge link of the weld image detection process finds a connection among ‘Found an image defect: 2021-07-04’, ‘Check for weld defects: 2021-07-04’, and ‘Inspecting of spinning machining quality: 2021-07-01’. It indicates that the defect in the weld image may be related to the workpiece in the spinning machining step. This implies that there may be a quality issue in the spinning machining of the previous process shell of welding. It also means our model can get the specific implicitly temporal knowledge links.

Table 5. Examples of specific temporal knowledge link reasoning found by our model.

Relationships (edges)	Temporal knowledge links reasoning
Perform a weld, not specified below: 2021-06-20	Predict to weld width: 2021-06-20 →Make an image detection: 2021-06-22 → Give to the decision: 2021-06-23
Make an image detection, not specified below: 2021-07-04	Found an image defect: 2021-07-04 → Check for weld defects: 2021-07-04 → Inspecting of spinning machining quality: 2021-07-01

5. Conclusion

In this paper, a semantic-aware event link reasoning over industrial knowledge graph embedding time-series data approach is proposed. It aims to mine and reason the implicit semantic information between massive dynamic time series data collected from IoT devices and technical documents data. What is more, we use NLTK to transform technical documents into an industrial knowledge graph. Considering the transformation of time series data into semantic information, a CNN-LSTM based product quality prediction method is employed. The prediction result is used as the semantic information of the predicted product at this moment. Further, the time series semantic information is associated with the industrial knowledge graph, which forms the industrial temporal knowledge graph with fused time-series semantics. Moreover, a TKEGN model based on graph embedding is presented to mine and reason the implicit information. Finally, we analyse the industrial temporal knowledge link reasoning task and achieve superior results than the baseline methods. However, root cause analysis for finding defects in the reasoning path may be limited and needs to be further researched.

Future work expects to collect more types of time-series data in the industry and construct global associations between different types of data from the semantic level. In turn, we also consider root cause analysis of failure issues in manufacturing to help engineers strengthen their skills in the cause-and-effect analysis of machining quality issues.

Acknowledgements

The authors wish to acknowledge an aerospace research institute for the precious collaboration. Also, the authors would like to express our sincere gratitude to the reviewers and the editor for their constructive comments and suggestions regarding this paper.

Data availability statement

The raw/processed data required to reproduce these findings cannot be shared at this time due to data privacy and security concerns restrictions in the aerospace company.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work is supported by the National Key Research and Development Program of China [grant number: 2019YFB1706300], and the Fundamental Research Funds for the Central Universities and Graduate Student Innovation Fund of Donghua University [grant number: CUSF-DH-D-2021043].

Notes on contributors

Bin Zhou

Bin Zhou is currently pursuing the Ph.D. degree in Mechanical Engineering from Donghua University, Shanghai, China. His main research interests are related to Industrial Knowledge Graph, Graph Neural Networks, Industrial Natural Language Processing, and Causality Analytics.

Xingwang Shen

Xingwang Shen is currently pursuing the Ph.D. degree in Mechanical Engineering from Donghua University, Shanghai, China. His research interests include Industrial Big Data Analytics and Knowledge Graph.

Yuqian Lu

Yuqian Lu is a Lecturer of Smart Manufacturing at the Department of Mechanical Engineering, The University of Auckland. He leads the Industrial Artificial Intelligence Research Group (https://iai.auckland.ac.nz). Prior to joining UoA in 2019, he was a Product Manager/R&D Team Lead at FRAMECAD Ltd. (2016–2019), Business Analyst at Compact Sorting Equipment Ltd. (2015–2016). His research interests span over Manufacturing Automation, Autonomous Systems, Industrial AI, and Robotics.

Xinyu Li

Xinyu Li currently works as a Research Professor at the College of Mechanical Engineering, Donghua University, Shanghai, China. He was a Research Fellow in Delta-NTU corporate laboratory for Cyber-Physical Systems, Nanyang Technological University, Singapore. He received a Ph.D. degree from Shanghai Jiao Tong University, Shanghai, China, in 2018, and a bachelor’s degree from Huazhong University of Science and Technology, Wuhan, China, in 2013. His research interests include Industrial Knowledge Graph, Smart Product-Service systems, and Cognitive Manufacturing.

Bao Hua

Bao Hua is currently pursuing the M.S. degree in Mechanical Engineering from Donghua University, Shanghai, China. His main research interests are in Knowledge Modeling, Knowledge Graph, and Deep Learning.

Tianyuan Liu

Tianyuan Liu received the Ph.D. degree in Mechanical Engineering from Donghua University, Shanghai, China, in 2021. His main research interests are Welding Automation, Industrial Image Processing, and Deep Learning.

Jinsong Bao

Jinsong Bao currently works as a Professor and Chairman of the Institute of Intelligent Manufacturing in the College of Mechanical Engineering, Donghua University, Shanghai, China. He is the Board Member of the Intelligent Manufacturing Professional Committee of the Chinese Society of Artificial Intelligence. He received the Ph.D. degree in Mechanical Engineering from Shanghai Jiao Tong University, Shanghai, China, in 2002, and a M.S. degree in Mechanical Engineering from Northeastern University, Shenyang, in 1998, respectively. His research interests include Intelligent Manufacturing Systems, Industrial Artificial Intelligence, Industrial Computer Vision, and Industrial Natural Language Processing. He is the author of three monographs, more than100 journal and conference papers, and 20 inventions. His research results won the Science and Technology Progress Award of Shanghai.

Notes

1 https://www.w3.org/TR/rdb-direct-mapping/.

2 http://d2rq.org/.

3 https://www.nltk.org/.

4 https://github.com/fxsjy/jieba.

5 https://github.com/thunlp/OpenKE; https://github.com/THUDM/cogdl.