342
Views
0
CrossRef citations to date
0
Altmetric
Research Article

ScTCN-LightGBM: a hybrid learning method via transposed dimensionality-reduction convolution for loading measurement of industrial material

ORCID Icon, , , &
Article: 2278275 | Received 13 Jul 2023, Accepted 27 Oct 2023, Published online: 09 Nov 2023

Abstract

Dynamic measurement via deep learning can be applied in many industrial fields significantly (e.g. electrical power load and fault diagnosis acquisition). Nowadays, accurate and continuous loading measurement is essential in coal mine production. The existing methods are weak in loading measurement because they ignore the symbol characteristics of loading and adjusting features. To address the problem, we propose a hybrid learning method (called ScTCN-LightGBM) to realize the loading measurement of industrial material effectively. First, we provide an abnormal data processing method to guarantee raw data accuracy. Second, we design a sided-composited temporal convolutional network that combines a novel transposed dimensionality-reduction convolution residual block with the conventional residual block. This module can extract symbol characteristics and values of loading and adjusting features well. Finally, we utilize the light-gradient boosting machine to measure loading capacity. Experimental results show that the ScTCN-LightGBM outperforms existing measurement models with high metrics, especially the stability coefficient R2 is 0.923. Compared to the conventional loading measurement method, the measurement performance via ScTCN-LigthGBM improves by 40.2% and the continuous measurement time is 11.28s. This study indicates that the proposed model can achieve the loading measurement of industrial material effectively.

1. Introduction

Loading measurement is a process of calculating the loading capacity of industrial materials (such as coal, sand, etc.), which is essential in energy utilization and enterprise efficiency. In coal mine production, we need to obtain the real-time loading capacity of truck carriages continuously and accurately. The conventional loading measurement method involves manual braking and static weighing, as shown in Figure . First, we brake the truck carriage manually through the speed control unit. Second, we pull the truck carriages completely onto the weighing unit with steel cables. It means that both the front and rear wheels of the carriage need to be on the track scale. Finally, we add up the pressure of carriage wheels to get the real-time loading capacity (i.e. FNR+ FNF in Figure ). However, this measurement method is discontinuous, laborious, and time-consuming, often causing material underload and economic disputes. Therefore, developing a continuous and accurate method of loading measurement is a challenge in coal mine production.

Figure 1. The weighing diagram of the industrial material load (e.g. FNR, FNR are the pressure value of the front and rear wheels on the track scale for each loading point).

Figure 1. The weighing diagram of the industrial material load (e.g. FNR, FNR are the pressure value of the front and rear wheels on the track scale for each loading point).

In artificial intelligence, hybrid learning models (Tuna et al., Citation2021) can achieve the target measurement in some industrial fields (e.g. electrical power load, and fault diagnosis acquisition). Some of these models (Wan et al., Citation2022; Xie et al., Citation2020) combine multi-learners or neural networks to improve the measurement effects, especially the convolutional network (CNN) with single dense layers or long short-term memory (Chen et al., Citation2021; Sheng et al., Citation2021). However, these measurement models may not perform well for sensitive time series tasks. The reason is that the fixed receptive field and network depth of CNN limit their ability to learn prior knowledge (Jiang et al., Citation2023).

To pursue wide receptive fields and high measurement accuracy, some researchers have explored hybrid measurement models via the temporal convolution network (TCN) (Diao et al., Citation2023; Wang et al., Citation2021). Based on the dilated causal convolution of TCN, these measurement models obtain flexible receptive fields (Yan et al., Citation2020). However, the TCN is an expansive CNN used to solve causal time-series measurement problems, which may not be suitable for the discrete time-series because the measurement value only relates to linear features of the current loading point.

In recent years, expansive decision trees have been widely introduced in hybrid measurement models, which are often combined with the TCN to measure discrete time-series data accurately (Chen et al., Citation2022). Especially, the gradient-boosting machine (GBM) has the advantages of fast convergence and easy cache training (Qiu et al., Citation2021; Wang et al., Citation2021). Based on the gradient-based one-side sampling (GOSS) and histogram algorithm of the GBM, the hybrid measurement models can improve measurement accuracy. However, loading and adjusting features have different normalized distributions (e.g. the truck speed is [0, 1], and the speed adjustment value is [−1, 1]). In a word, the existing measurement models cannot deal with symbol characteristics of different features well in the loading measurement process.

To overcome the shortage of symbol characteristics extraction in the loading measurement of industrial material, this paper proposes a ScTCN-LightGBM based on a sided-composited temporal convolutional network (ScTCN) and the light-gradient boosting machine (LightGBM). Compared to existing measurement models, the proposed model can extract loading and adjusting features well and measures loading capacity accurately. The advantage is that the ScTCN integrates the novel transposed dimensionality-reduction convolution residual blocks (TRBs) with conventional residual blocks (RBs), which builds a wide-composited receptive field to extract symbol characteristics and values of different normalized features. Further, we adopt the Light-GBM to measure the loading capacity accurately and quickly. Experimental results show that the ScTCN-LightGBM outperforms existing measurement models with high evaluation metrics (i.e. MAPE: 9.17%, RMSE: 0.054, MAE: 0.046, and R2: 0.923). Compared to the conventional loading measurement method, the performance via ScTCN-LigthGBM improves by 40.2% and the continuous measurement time is 11.28s.

The main contributions of the paper are as follows:

  1. We propose a novel transposed dimensionality-reduction convolution structure attached to the conventional residual block (RB), called TRB. Further, we replace the convolutional shortcut with the pure shortcut, which is appointed as the main branch of TRB.

  2. We combine sided-TRBs with RBs as a new composited structure to replace TCN and LSTM in existing measurement models, which extracts symbol characteristics and values of loading and adjusting features well, called ScTCN.

  3. Unlike the conventional loading measurement method, we propose a ScTCN-LightGBM to achieve the loading measurement of industrial material continuously and intelligently. This method adopts the ScTCN module to extract loading and adjusting features of each loading point (i.e. Table ). Further, we utilize the Light-GBM to measure the real-time loading capacity of industrial material, replacing manual weighting on the track scale.

Table 1. The related dynamic parameters and data processing methods (single loading point).

The rest of the paper is organized as follows. Section 2 gives a brief review of related works. Section 3 introduces the structure of ScTCN-LightGBM. Section 4 gives the experimental results and theoretical analysis. Finally, the conclusion and future work are given in Section 5.

2. Related works

2.1. Hybrid networks for industrial target measurement

The hybrid networks for industrial target measurement are divided into convolutional-temporal networks (Chen et al., Citation2021; Li et al., Citation2022) and hybrid temporal networks (Bi et al., Citation2022; Jiang et al., Citation2023). Hybrid convolutional-temporal networks often combine the convolutional neural network (CNN) with the recurrent neural network (e.g. LSTM, GRU) to achieve accurate target measurement. Ren et al. (Citation2021) propose a model based on CNN-GRU to identify rough-stored express deliveries. Jalali et al. (Citation2022) propose a hybrid model that combines CNN and LSTM for wind power measurement. Wang et al. (Citation2023a) and Sun and Fan (Citation2023) adopt the CNN-LSTM to measure the surface failure of metals. These models integrate the feature aggregation for CNN and cell selection memory for expansive LSTM, which obtains excellent advantages in related measurement processes. However, the fixed receptive field limits the feature extraction of long-time series, which will result in model overfitting and low generalization ability.

Further, hybrid temporal networks that combine TCN and LSTM are provided to improve the receptive field and feature extraction ability. Yang et al. (Citation2022) adopt the multi-TCN and attention-LSTM to measure the electrical short-term price accurately. Ehteram and Ghanbari-Adivi (Citation2023) provide a groundwater level measurement model based on self-attention TCN and LSTM. Further, Mo et al. (Citation2023) propose a hybrid-TCN model for effective power load measurement. However, due to the lack of extraction accuracy for features’ symbol characteristics, they may not be suited for the loading measurement of industrial material.

2.2. Expansive decision trees for industrial target measurement

Expansive decision trees are popular in many industrial measurement fields. The expansive decision trees mainly include the gradient-boosting decision tree (GBDT) and the GBM.

The GBDT adopts multi-CART trees in series to achieve industrial target measurement, while the GBM utilizes parallel local optimum and one-sided split suppression for improving performance. For example, Ju et al. (Citation2019) propose a convolutional neural network and a gradient-boosting machine for wind power measurement. Luo and Chen (Citation2020) provide strong boosting predictors based on GBDT to measure the state of blast furnace heat. Lu et al. (Citation2021) perform hybrid learning of error estimates for resonant circuits based on a traditional mathematical model and the GBDT. Wang et al. (Citation2023b) propose a Bayesian optimization-GBDT model to achieve evapotranspiration in the agricultural IoT. These models via GBDT can process low-dimensional series data with high accuracy while being less effective for high-dimensional parallel data. Further, Qiu et al. (Citation2021) provide a measurement model via the edge-weighted network and GBM, which applies to industrial machines of the IoT. Wang et al. (Citation2021) provide a TCN-LightGBM model to measure the short-term load of industrial power systems accurately. Hussien et al. (Citation2023) propose the multi-GBM model for the measurement of carrier frequency offset, which obtains a good generalization behaviour.

To the best of our knowledge, the hybrid measurement models based on neural networks and expansive decision trees are less mentioned in the industrial loading field. Thus, we will explore the specific structure of the ScTCN-LightGBM in Section 3.

3. ScTCN-LightGBM architecture

In this paper, we propose a continuous loading measurement via ScTCN-LightGBM. The idea of this method is quite simple. Firstly, the truck carriage is divided into a fixed number of loading points, such as 100. Second, we get the loading and adjusting data of each loading point by industrial sensors (e.g. displacement sensors, weight sensors, etc.). Third, the data processing methods are employed to supplement and normalize the raw dataset (Zhang et al., Citation2022; Zhang et al., Citation2023). Finally, we adopt the ScTCN-LightGBM to measure the loading capacity of industrial material continuously.

Notably, the structure of the ScTCN-LightGBM consists of three parts: the data preparation and processing, the fusion feature extraction based on the ScTCN, and the gradient-optimized measurement via the Light-GBM, as shown in Figure .

Figure 2. The framework of the ScTCN-LightGBM model.

Figure 2. The framework of the ScTCN-LightGBM model.

3.1. Data preparation and processing

This paper focuses on the truck carriage with a standard load (70 tons). We divide the carriage into 100 loading points and collect related features and labels from each point. The raw dataset comes from the daily loading measurement process in real coal mines. However, manual interference may cause some exception data (i.e. missing or low-precision data) in loading and adjusting features. Thus, according to the “ Loading and Measurement Instruction of Intelligent Coal System” (made in the Huaibei Coal Mine) and actual scene survey, we develop related data collection deviations and supplement methods to ensure dataset accuracy, as shown in Table .

Further, to eliminate measurement item effects between different training features, we adopt a series of data normalization methods (i.e. MinMaxscaler and MaxAbsScaler), as denoted in Formula (1) and Formula (2) respectively. However, the numerical normalization methods fail to accurately handle categorical labels and specific string identifiers. Thus, we select the LabelBinarizer (Semmelmann et al., Citation2022) to convert the truck carriage item (e.g. No. 0622220) into a one-hot encoding (e.g. 000,001), as shown in Formula (3). (1) Xloading=Xloading(Xloading)min[(Xloading)max(Xloading)min,axis=0](1) (2) Xadjusting=X^adjustingMax(abs(X^adjusting),axis=0)(2) (3) Numone-hot[Num0,Num1,,NumK][0_0_0_1_0010001000]K×K(3) where Xloading=[Num,D,V,Q,T] is the input of loading data (i.e. axis = 0), (Xloading)max is the maximum row vector for Xloading, (Xloading)min is the minimum row vector, and Xloading is the normalized input of loading data. Xadjusting=[H,ΔV,ΔQ] is the adjusting data input. abs(X^adjusting) represents the absolute value of the column vector X^adjusting, and Xadjusting is the normalized input of loading data. Num0toK are the one-hot codes of the truck carriage item. K is the number of different truck carriages (i.e. K = 6).

After data processing and normalization, we obtain the standard training dataset of loading and adjusting data. The loading features have a normalized distribution between [0, 1] (e.g. the truck speed in Figure (a)), and that of adjusting features are between [−1, 1] (e.g. the speed adjusting value in Figure (b)).

Figure 3. The diagram of the loading features and the adjusting features.

Figure 3. The diagram of the loading features and the adjusting features.

3.2. Feature extraction based on the ScTCN

The sided-composited temporal convolutional network (ScTCN) is a composited feature extraction module that combines TRBs and RBs. The module is divided into three parts: the conventional residual block, the transposed dimensionality-reduction convolution residual block, and the final extraction matrix construction, as shown in Figure .

Figure 4. The ScTCN extraction module.

Figure 4. The ScTCN extraction module.

3.2.1. Conventional residual block (RB)

The residual block is developed to preserve the original input information. The conventional residual block in TCN includes three parts: the dilated causal convolution, the Batch normalization, and the Relu layer. The dilated causal convolution can strengthen the generalization ability of long-history information effectively. The Batch normalization & Relu layer are used to speed up the training process and solve gradient loss problems.

In ScTCN, we utilize the RB to extract loading features and avoid the gradient vanishing. It also can improve the extraction ability of the-sided TRB. Namely, while the TRB focuses on the symbol extraction of loading and adjusting features, its ability to generalize hidden feature relationships may be constrained. Based on the analysis, we combine the RB and the sided-TRB as a composited structure in ScTCN.

In RB, the raw input of loading features is a one-dimensional vector Xloading=(x0,x1,,xt,,xT).. After a series of transformation operations, the dilated causal convolution FRB() for each element T can be denoted in Formula (4). Further, a transformation branch (e.g. convolution, normalization, and Relu activation layers) is connected to the convolutional shortcut as a single RB for extracting loading features, as denoted in Formula (5). Based on Formula (4) and (5), we can get the final output of m-conventional RBs. Also, the zero-padding method is adopted to make the output size of RB consistent with that of TRB, as denoted in Formula (6). (4) FRB(T)=(Xloadingdfi)(T)=i=0k1f(i)xTdi,i=0,1,..,k1(4) (5) HRB_l(X)=Add[Xc_l1,FRB_l(T)](5) (6) YRB=[HRB_1[X1,X2,,Xloading,0,,0],HRB_2,,HRB_m]=[x0,x1,,x5,,xn1,0,,0]T(6) where k is the kernel size, d is the dilated factor, and Tdi is the past direction. f()={0,1,,n1} denotes the convolutional operation of the i-th kernel. (x0,x1,,x5) is the feature vector of the truck number. FRB_l() denotes a series of convolution, normalization, and Relu activation of l-th RB, HRB_l(X) dentoes the output of l-th RB, and Xc_l1 denotes the side convolutional shortcut of l-th RB. YRB=[HRB_1,HRB_2,,HRB_m] is the composite of feature propagation maps produced in m-RBs, and n is the length of feature extraction vectors.

3.2.2. Transposed dimensionality-reduction convolution residual block (TRB)

Recently, the conventional RBs (i.e. as shown in Figure a) with an intensive shortcut have been widely adopted in optimizing the model’s generalization ability. This structure can suppress the degradation of the deep network through the loss-less propagation of shortcut connections. The bottleneck layer has the advantage of two-dimensional feature extraction (Park & Shin, Citation2022). It reduces the feature dimensionality of filters to retain essential reconstruction features, which can be suitable for symbol characteristics extraction. Further, the up-sampled operations can amplify high-level features and avoid information loss in the convolution process (e.g. the transposed convolution (Im et al., Citation2020; Peralta & Saeys, Citation2020)).

  1. We appoint the pure shortcut as the main branch while considering the novel transposed dimensionality-reduction convolution branch as a side path in TRB. We discard the 1 × 1 convolutional operation and adopt pure shortcut as the main branch. It can ensure the propagation purity of features in the TRB.

  2. We propose a transposed dimensionality-reduction convolution attached to the conventional RB to extract significant features intuitively. The transposed dimensionality-reduction convolution is a combined structure, including a transposed convolution and the conventional convolution with a dimensionality-reduction ratio. First, we use a transformation layer to flatten features between two dilated causal convolution layers of the RB. Second, we design a novel transposed convolution layer with a b-ratio dimensionality reduction to enhance the symbolic features. With the help of the transposed convolution and filter-dimensionality reduction, the TRB can find hidden relationships of features. Third, the 1 × 1 convolution is adapted to match the last dimensionality of the main branch. Further, we swap the convolutional layer and the fully pre-activation layer to reduce gradient overfitting (i.e. momentum = 0.9, alpha = 0.3) (Esmaeilzehi et al., Citation2021). Finally, the outputs of TRB are the simple sum of each residual branch. The main operations of TRB can be described in Formula (7) to (10). (7) Xtrans=CTC([Xloading,Xadjusting]dfi)(T)(7) (8) Xreduce=k=0cout1weight(cout,k)Xtrans,k+bais(cout),cout=bc(8) (9) HTRB_l=Add[Xl1,FTRB_l(Xreduce)](9) (10) YTRB=[HTRB_1[X1,X2,,Xloading\& adjusting],HTRB_2,,HTRB_m]=[x0,x1,x5,x6,,xn1,xn,xn+1]T(10) where C is the sparse matrix (i.e. according to the convolution kernel size k), CT is the transposed sparse matrix. Xtrans denote transposed features of loading and adjusting data. Xreduce denote the dimensionality-reduction convolutional features. FRB_l() denotes a series of dilated causal convolution, normalization, and Relu activation operations of l-th TRB. Xl1 denotes the pure shortcut of l-th RB. HTRB_l dentoes the output of l-th RB. YTRB=[HTRB_1,HTRB_2,,HTRB_m] is the composite of feature propagation maps produced in m-TRBs.

3.2.3. Final extraction matrix construction

In the output of the ScTCN module, both the feature and symbol characteristics extracted by RBs and TRBs are essential to the loading measurement of industrial material. We discard the activation layer (e.g. Relu activation) to prevent feature degradation. The reconstructed feature matrix Yfinal after the Dense flattened operation can be denoted in Formula (11). (11) Yfinal=Add[YRB,YTRB]|Dense(Dropout)[(x0+x0)/2,,(x5+x5)/2,(x6+x6)/2,,(xn1+xn1)/2,xn/2,xn+1/2,xn+2/2](11) where Yfinal is the final feature output of the fusion loading and adjusting data via the ScTCN. xi is the element of the feature extraction vector RBs, and xi is the element of the feature extraction vector TRBs.

Figure 5. The structure of residual blocks. (a) is the convolutional RB. (b) is the TRB (stride = 1). Inspired by the ideas, we provide a TRB in ScTCN to extract symbol characteristics of loading and adjusting features effectively, as shown in Figure b. The detailed structure of the TRB is as follows.

Figure 5. The structure of residual blocks. (a) is the convolutional RB. (b) is the TRB (stride = 1). Inspired by the ideas, we provide a TRB in ScTCN to extract symbol characteristics of loading and adjusting features effectively, as shown in Figure 5b. The detailed structure of the TRB is as follows.

3.3. The gradient-optimized measurement via the light-GBM

Normally, we increase the depth of the network layers to strengthen the forecasting performance of the corresponding model. However, it will sacrifice the computational time and training gradient. Interestingly, many machine learning models (e.g. expansive decision trees) can achieve superior forecasting effects with simple multi-weaker learners, especially the light-gradient boosting machine (Light-GBM).

The Light-GBM is a learning method via the gradient boosting decision trees, which minimize the gradient loss by integrating multi-weak learners. This module can process large amounts of data with less consumption, which is suitable for measuring the loading capacity of industrial material. Therefore, we adopt the Light-GBM as the loading measurement module.

  1. We rank all training data in descending order and select the top a×100% data as the subset P. From the rest of the data, we randomly select b×100% data as the subset Q. Based on the gradient-based one-side sampling method (GOSS), the estimated variance gain Gj(v) of the splitting feature oj at point v is denoted in Formula (12) to (14). (12) Gj(v)=1m(Gj1(v)+Gj2(v))(12) (13) Gj1(v)=1ml(v)(OkPlgk+1abOkPlgk)2(13) (14) Gj2(v)=1mt(v)(OkPtgk+1abOkPtgk)2(14) where m is the number of the subset PQ. ml(v) and mt(v) denote the number that the feature oj is less than or more than v, respectively.Pl={OkP:Okjv},Ql={OkQ:Okjv},Ph={OkP:Okj>v}, and Qh={OkP:Okj>v}.

  2. Further, we adopt the histogram algorithm and the leaf-wise strategy of the Light-GBM to find a depth-limited leaf with the largest splitting gain. The gradient-optimized gain function adopts the function gt(x), as denoted in Formula (15). Further, the histogram is utilized to simplify the objective gradient and get the minimum gain value, as denoted in Formula (16). The final loading measurement output via the Light-GBM module is denoted in Formula (17). (15) gt(x)=Ey[Ψ(y,f(x))f(x)|x]f(x)=f^t1(x)(15) (16) (ρt,θt)=argminρ,θi=1N[gt(xi)+ρh(xi,θ)]2(16) where {gt(xi)}i=1M is the negative gradient, Ψ(y,f(x)) denotes the specific loss function, h(x,θ) is a custom base-learner function, and p denotes the boundary expansion. f(x)=f^t1(x) is the (t-1)-th function estimation. (ρ,θ) denotes the optimization parameters (i.e. the step size and the functional dependence parameter). (17) M^=Yfinal(o)+λ=1T[νλu=1Jcλ,uI(oλ,u)](17) where M^ is the measurement value of the loading capacity. Yfinal(o) is the initial input. If o∈leafm,j, I = 1; else I = 0. T is the number of trees, λ,u is the leaf node area of the λ-th decision tree, u = {1, 2, … , J } is a leaf node, and v denotes a zoom factor, and cλ,u is the minimal residual loss value of the leaf node u in the λ-th decision tree.

3.4. Time complexity analysis

This loading measurement of industrial material via the ScTCN-LightGBM is summarized in Algorithm 1, which can be described as follows.

Algorithm 1 The loading measurement of industrial material via ScTCN-LightGBM

  1. Step 1–6 denotes the data processing and extraction of industrial adjusting and loading features via the ScTCN module. The time complexity of the feature extraction model via the ScTCN is about O(NB1K+NB2K). N is the number of residual blocks, B is the time complexity of convolutional operation in each block, and K is neural network training epoch.

  2. Step 7–15 denotes the loading measurement via Light-GBM and some related metrics evaluation. The time complexity is about O(JM+J×[T×length[(a%)+(b%)]×D]). J is the training data, M is the number of bins, T×length[(a%)+(b%)] is the length of features, and D is the max-depth of trees. In a word, the time complexity of the ScTCN-LightGBM model is the sum of all modules.

4. Experiments

4.1. Experimental settings and metrics

In this paper, we adopt actual loading datasets collected from the coal mines of Anhui Province, China. The experimental object is a single carriage with a standard loading capacity (70 tons), as shown in Figure . The experimental programming environments are the Python 3.9 programming environment, the Keras framework, the NVIDIA RTX 3090 GPU, the AMD R7-5800x CPU, and 32 GB of memory.

Figure 6. The schematic of material loading capacity.

Figure 6. The schematic of material loading capacity.

Further, we select the determination coefficient (R2), the Root Mean Square Error (RMSE), the Mean Absolute Error (MAE), and the Mean Absolute Percentage Error (MAPE) as models’ evaluation metrics. (17) RMSE=1Jz=1J(MzM^z)2(17) (18) MAE=1J|z=1J(MzM^z)|(18) (19) MAPE=1Jz=1J|(MzM^z)Mz|×100%(19) (20) R2=1SSESST=1z=1J(M^zMz)2z=1J(M¯Mz)2(20) where M^z and Mz are the measurement and actual loading capacity, respectively. M¯ is the average of the actual loading values Mz.

4.2. Experimental results for the dimensionality reduction

In this paper, to explore the extraction performance of the ScTCN, we compare the ScTCN-LightGBM with different dimensionality reduction ratios. The historical loading and adjusting data come from Jun 1st, 2022 to Sept 1st, 2022 in the Huaibei Mining Co., Ltd, about 20,000 items. This dataset is split into the training set and the testing set according to the proportion of 8:2.

Further, the parameter setting is described as follows. The training epoch of the ScTCN is 500, the kernel size is 3, the learning rate is 0.001, and the filter factors are 64/64/128/128. In TRBs and RBs, the dilated causal convolution branch is [1, 2, 4, 8], the dimensional reduction ratios are [1/2; 1/4; 1/8; 1; 2; 4], the chosen optimizer is Adam, and the fully connected layer is 13/1. Further, the trees of the Light-GBM are 500, the maximum depth is 6, the learning rate is 0.01, and the boosting method is GBDT.

Figure (a) presents the training loss of the ScTCN-LightGBM model with different ratios. Figure (b) is the fitted curve of the Figure (a). We know that the ScTCN with the dimensionality reduction ratio b = 1/2 can get the best performance. In Table , the effect of the ScTCN becomes weaker when the dimensionality reduction ratio decreases from 1/2–1/8. Further, the effect also becomes weaker when the dimensionality reduction ratio increases from 1/2–4. When the ratio is b = 1/2, the ScTCN-LightGBM can get topgallant evaluation metrics (i.e. the RMSE about 0.054, the MAE about 0.046, the R2 about 0.928, and the MAPE about 9.67%). In conclusion, we finally select b = 1/2 as the dimensionality reduction ratio of the ScTCN.

Figure 7. The training loss of the ScTCN-LightGBM.

Figure 7. The training loss of the ScTCN-LightGBM.

Table 2. Results of the model with different ratios.

4.3. Experimental results for each sub-module of the ScTCN-LightGBM

To explore the real performance of each module in the proposed ScTCN-LightGBM, we compare the ScTCN*-LightGBM (i.e. non-RB in the ScTCN), the ScTCN-FC (i.e. the ScTCN with the fully connected layer), the LightGBM model with the ScTCN-LightGBM. The main structures of contrast models are shown in Figure .

Figure 8. The main structure of contrast models. (a) ScTCN-FC; (b) ScTCN*-LightGBM.

Figure 8. The main structure of contrast models. (a) ScTCN-FC; (b) ScTCN*-LightGBM.

The results of each contrast module are listed in Table . First, compare the ScTCN-FC and the Light-GBM to the proposed ScTCN-LightGBM, the comprehensive metrics are reduced by 15.5% and 27.6%, respectively. Second, the comprehensive metrics of the ScTCN*-LightGBM are reduced by 3.98%, which demonstrates the superior measurement effects of the TRB in the ScTCN. Finally, we know that both the ScTCN and the Light-GBM can improve the performance of the ScTCN-LightGBM, but the ScTCN plays a more significant role than the LightGBM.

Table 3. The results of each contrast module.

4.4. Ablation experiment for each unit of the TRB

To further explore the effect of each unit of the TRB, we compare the TRB without the transposed convolution in the ScTCN (i.e. called the ScTCN+), the TRB without the dimensional reduction convolution in the ScTCN (i.e. called the ScTCN++) with the RB-LightGBM. Further, the main parameters of each model are described in Table .

Table 4. The detailed parameters of each model.

Table  shows the contrast effects of each unit of the TRB. Among them, the comprehensive performance of the ScTCN+ (3.98%↓) and the ScTCN++ (9.49%↓) is superior to that of the RBs (20.2%↓), while the ScTCN++-LightGBM is weaker than the ScTCN + -LightGBM. It indicates that both the transposed convolution and dimensional reduction convolution improve the measurement accuracy, but the dimensional reduction convolution plays an important role in TRB.

Table 5. The measurement results of each sub-unit of the proposed model.

4.5. Experimental results for the contrast models

In this paper, we compare the existing measurement models of related fields with the proposed ScTCN-LightGBM, including CNN-LSTM (Ren et al., Citation2021), the TCN-LSTM (Bi et al., Citation2022), the TCN-LightGBDT, the CNN-LightGBDT, the CNN-LightGBM (Ju et al., Citation2019), and the TCN-LightGBM (Wang et al., Citation2021). The detailed parameters are listed in Tables  and . In the experiment, we train three times to eliminate the influence of the lucky network and the comparison results are shown in Tables  and .

Table 6. The parameters of each extraction module .

Table 7. The parameters of each measurement module.

Table 8. Computational complexity of each contrast model.

Table 9. Experimental results of each contrast model.

The proposed ScTCN-LightGBM can achieve the best evaluation metrics than other measurement models (i.e. MAPE: 8.89%, RMSE: 0.054, MAE: 0.046, and R2: 0.928). Further, the computational time of 100 continuous loading points via the ScTCN-LightGBM is about 0.33s. Namely, the computational time for a single loading point is 0.003s. It can be accepted in the actual industrial loading process while considering the improved measurement accuracy.

4.6. Experiment for the effect of ScTCN-LightGBM

To verify the measurement effect of the ScTCN-LightGBM, we compare above above-listed measurement models with the proposed ScTCN-LightGBM in another dataset. This dataset is collected in the Linhuan Coal Preparation Plant from Aug 1st, 2022 to Nov 1st, 2022, with about 20,000 data. The parameters of each extraction module are summarized in Table .

Table  verifies that the ScTCN-LightGBM can get the topgallant performance compared to existing measurement models of related fields (i.e. MAPE: 9.17%, RMSE: 0.054, MAE: 0.046, and R2: 0.923). Further, the computational time of the ScTCN-LightGBM is similar to the data in Table , about 0.31s. In conclusion, the ScTCN-LightGBM can be well-suitable for the loading measurement of industrial material.

Table 10. The parameters of each contrast model.

Table 11. Verification results of all measurement models.

4.6. Comparison of existing loading measurement methods

To demonstrate the effect of loading measurement via ScTCN-LightGBM, we perform a simple comparison experiment with the conventional loading measurement method. The specific results are shown in Figure  and Table .

Figure 9. The real effects of material loading capacity measurement.

Figure 9. The real effects of material loading capacity measurement.

Table 12. Comparison results of existing loading measurement methods.

First, the proposed method presents low-error measurement effects of each loading point, while the conventional method shows significant oscillatory errors (i.e. RMSE: 0.054 vs. 0.082; R2: 0.921 vs. 0.498). Second, the total measurement time of the manual static weighing method is about 10 min, while that of the proposed method is only 11.28s (e.g. model start-up time:10.93s, measurement time: 0.35s). Finally, our proposed method via the ScTCN-LightGBM can measure the total loading capacity of the carriage accurately with the error of 0.09 tons. In general, compared to the conventional measurement method, the proposed method via the ScTCN-LightBGM improves the comprehensive performance by 40.2%, and the total measurement time is only 11.28s.

4.7. Discussion and analysis

This paper proposes a ScTCN-LightGBM for continuous loading measurement. The experimental results prove that the proposed model outperforms the listed measurement models. Some insightful analyses are provided as follows:

  1. The ScTCN has a better extraction effect of loading and adjusting features than the CNN and the TCN. In Table , we calculate the size of the receptive field for the different convolution models (Choudhary & Sharma, Citation2023; Zheng et al., Citation2023). The receptive field of CNN is 2 (Fan et al., Citation2023). The receptive field of the TCN is significantly larger than that of CNN (i.e. the TCNmax: 8). Further, the receptive field of ScTCN is larger than the TCN (i.e. the TCNmain: 1/2/4/8, and the ScTCNmain: 5/9/13/17), which makes the ScTCN get more superior performance than the CNN and TCN. The receptive field can be calculated in Formulas (21) and (22). (21) rc=rc1+[(kc1)l=1c1sl](21) (22) ωl=1+i=0l1(k1)bi=1+(kl1)bl1b1(22) where rc is the receptive field size of the c-th convolutional layer, kc is the kernel size of the c-th layer or the pooling layer size, and sl is the multiplication of the convolutional strides of the previous (c-1)-th layers. Also, ωl represents the receptive field size of the l-th dilated residual layer. kl represents the kernel size of the l-th layer, and b represents the dilated factor (i.e. b = 2).

  2. The Light-GBM has better measurement performance than the Light-GBDT and the LSTM. Due to the gradient-based one-side sampling method and the histogram algorithm (Khan et al., Citation2021; Wen et al., Citation2021), the optimized Light-GBM obtains better measurement performance than the Light-GBDT and the LSTM. Further, the Light-GBM/GBDT combined with other models needs to reduce the dimensional channel, which can obtain a low time complexity of loading measurement of industrial material.

Table 13. Receptive field size of the CNN, the TCN, and the TRB.

In conclusion, we find from Tables  and that the computational time is about 0.35s per 100 samples (e.g. 0.003s). In Table , the loading time of a loading point is about [1.50, 3.00]s. The proposed model via ScTCN-LightGBM can achieve real-time loading measurement of industrial material effectively.

5. Conclusion

This paper is academic research based on actual industrial demands. We propose a ScTCN-LightGBM model via the ScTCN and the Light-GBM to achieve loading measurement of industrial material continuously and accurately. The two contributions are summarized as follows.

  1. Loading and adjusting features via the ScTCN is effectively extracted. The novel transposed dimensionality-reduction convolution residual blocks are combined with conventional residual blocks in the ScTCN to extract symbol characteristics and values of adjusting and loading features.

  2. Loading measurement of industrial material is successfully achieved via the ScTCN-LightGBM. We propose a loading measurement via ScTCN-LightGBM to measure the real-time loading capacity of each loading point continuously and accurately. Experimental results demonstrate the superior performance of ScTCN-LightGBM, especially the stability R2 is 0.923. Compared to the conventional loading measurement method, the measurement performance via ScTCN-LigthGBM improves by 40.2% and the continuous measurement time is 11.28s.

However, there are still some limitations that have not yet been considered. For example, the research only employs the truck carriage with a standard loading capacity (70 tons). In the future, we will explore a better feature extraction module for achieving accurate loading measurements with multi-standard loading capacities (e.g. 50 and 60 tons). We further advise researchers to try to integrate the adaptive dilated causal convolution with expansive GBDT to achieve accurate measurement in other related fields.

Declarations

This article is for non-life science journals.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the Anhui Simulation Design and Modern Manufacturing Engineering Technology Research Center Open Subject [Grant Number SGCZXZD2101]; the Provincial Natural Science Research Projects in Anhui Universities (A study of life detection methods for two-dimensional optimization of MIMO array signals in space and time) [Grant Number KJ2021JD22].

Reference

  • Bi, J., Zhang, X., Yuan, H., Zhang, J., & Zhou, M. (2022). A hybrid prediction method for realistic network traffic with temporal convolutional network and LSTM. IEEE Transactions on Automation Science and Engineering, 19(3), 1869–1879. https://doi.org/10.1109/TASE.2021.3077537
  • Chen, X., Zhang, B., & Gao, D. (2021). Bearing fault diagnosis base on multi-scale CNN and LSTM model. Journal of Intelligent Manufacturing, 32(4), 971–987. https://doi.org/10.1007/s10845-020-01600-2
  • Chen, Z., Wang, C., Li, J., Zhang, S., & Ouyang, Q. (2022). Multi-agent collaborative control parameter prediction for intelligent precision loading. Applied intelligence, 52, 15961–15979. https://doi.org/10.1007/s10489-022-03297-7
  • Choudhary, S., & Sharma, R. (2023). CNN-based Battlefield classification and camouflage texture generation for real environment. International Journal of Computational Science and Engineering, 26(3), 231–242. https://doi.org/10.1504/IJCSE.2023.131514
  • Diao, C., Zhang, D., Liang, W., Li, K., Hong, Y., & Gaudiot, J. (2023). A novel spatial-temporal multi-scale alignment graph neural network security model for vehicles prediction. IEEE Transactions on Intelligent Transportation Systems, 24(1), 904–914. https://doi.org/10.1109/TITS.2022.3140229
  • Ehteram, M., & Ghanbari-Adivi, E. (2023). Self-attention (SA) temporal convolutional network (SATCN)-long short-term memory neural network (SATCN-LSTM): An advanced python code for predicting groundwater level. Environmental Science and Pollution Research, 30, 92903–92921. https://doi.org/10.1007/s11356-023-28771-8
  • Esmaeilzehi, A., Ahmad, M. O., & Swamy, M. N. S. (2021). Srnharb: A deep light-weight image super resolution network using hybrid activation residual rlocks. Signal Processing: Image Communication, 99, 116509. https://doi.org/10.1016/j.image.2021.116509
  • Fan, J., Zhang, K., Huang, Y., Zhu, Y., & Chen, B. (2023). Parallel spatio-temporal attention-based TCN for multivariate time series prediction. Neural Computing & Applications, 35(18), 13109–13118. https://doi.org/10.1007/s00521-021-05958-z
  • Hussien, M., Abdelmoaty, A., Elsaadany, M., Ahmed, M., Gagnon, G., Nguyen, K., & Cheriet, M. (2023). Carrier frequency offset estimation in 5G NR: Introducing gradient boosting machines. IEEE Access, 11, 34128–34137. https://doi.org/10.1109/ACCESS.2023.3263053
  • Im, D., Han, D., Choi, S., Kang, S., & Yoo, H. (2020). DT-CNN: An energy-efficient dilated and transposed convolutional neural network processor for region of interest based image segmentation. Ieee Transactions on Circuits and Systems I-Regular Papers, 67(10), 3471–3483. https://doi.org/10.1109/TCSI.2020.2991189
  • Jalali, S. M. J., Osorio, G. J., Ahmadian, S., Lotfi, M., Campos, V. M. A., Shafie-khah, M., Khosravi, A., & Catalao, J. P. S. (2022). New hybrid deep neural architectural search-based ensemble reinforcement learning strategy for wind power forecasting. IEEE Transactions on Industry Applications, 58(1), 15–27. https://doi.org/10.1109/TIA.2021.3126272
  • Jiang, W., Ling, L., Zhang, D., Lin, R., & Zeng, L. (2023). A time series forecasting model selection framework using CNN and data augmentation for small sample data. Neural Processing Letters, https://doi.org/10.1007/s11063-022-11113-z
  • Ju, Y., Sun, G., Chen, Q., Zhang, M., Zhu, H., & Rehman, M. U. (2019). A model combining convolutional neural network and LightGBM algorithm for ultra-short-term wind power forecasting. IEEE Access, 7, 28309–28318. https://doi.org/10.1109/ACCESS.2019.2901920
  • Khan, Y. A., Shan, Q. S., Liu, Q., & Abbas, S. Z. (2021). A nonparametric copula-based decision tree for two random variables using MIC as a classification index. Soft Computing, 25(15), 9677–9692. https://doi.org/10.1007/s00500-020-05399-1
  • Li, W., Wei, Y., An, D., Jiao, Y., & Wei, Q. (2022). LSTM-TCN: Dissolved oxygen prediction in aquaculture, based on combined model of long short-term memory network and temporal convolutional network. Environ Sci Pollut Res, 29, 39545–39556. https://doi.org/10.1007/s11356-022-18914-8
  • Lu, X., Yang, S., & Mao, S. (2021). Hybrid modeling for LCC circuit based on machine learning. 2021 IEEE 1st International Power Electronics and Application Symposium (PEAS), 1–4. https://doi.org/10.1109/peas53589.2021.9628776
  • Luo, S., & Chen, T. (2020). Two derivative algorithms of gradient boosting decision tree for silicon content in blast furnace system prediction. IEEE Access, 8, 196112–196122. https://doi.org/10.1109/ACCESS.2020.3034566
  • Mo, J., Wang, R., Cao, M., Yang, K., Yang, X., & Zhang, T. (2023). A hybrid temporal convolutional network and prophet model for power load forecasting. Complex & Intelligent Systems, 9, 4249–4261. https://doi.org/10.1007/s40747-022-00952-x
  • Park, S., & Shin, Y. G. (2022). Generative residual block for image generation. Applied Intelligence, 52(7), 7808–7817. https://doi.org/10.1007/s10489-021-02858-6
  • Peralta, D., & Saeys, Y. (2020). Robust unsupervised dimensionality reduction based on feature clustering for single-cell imaging data. Applied Soft Computing, 93, 106421. https://doi.org/10.1016/j.asoc.2020.106421
  • Qiu, T., Zhang, M., Liu, X., Liu, J., Chen, C., & Zhao, W. (2021). A directed edge weight prediction model using decision tree ensembles in industrial internet of things. IEEE Transactions on Industrial Informatics, 17(3), 2160–2168. https://doi.org/10.1109/TII.2020.2995766
  • Ren, L., Dong, J., Wang, X., Meng, Z., Zhao, L., & Deen, M. J. (2021). A data-driven auto-CNN-LSTM prediction model for lithium-Ion battery remaining useful life. IEEE Transactions on Industrial Informatics, 17(5), 3478–3487. https://doi.org/10.1109/TII.2020.3008223
  • Semmelmann, L., Henni, S., & Weinhardt, C. (2022). Load forecasting for energy communities: A novel LSTM-XGBoost hybrid model based on smart meter data. Energy Informatics, 5. https://doi.org/10.1186/s42162-022-00212-9
  • Sheng, Z., Wang, H., Chen, G., Zhou, B., & Sun, J. (2021). Convolutional residual network to short-term load forecasting. Applied Intelligence, 51(4), 2485–2499. https://doi.org/10.1007/s10489-020-01932-9
  • Sun, H., & Fan, Y. (2023). Fault diagnosis of rolling bearings based on CNN and LSTM networks under mixed load and noise. Multimedia Tools and Applications, https://doi.org/10.1007/s11042-023-15325-w
  • Tuna, T., Beke, A., & Kumbasar, T. (2021). Correction to: Deep learning frameworks to learn prediction and simulation focused control system models. Applied Intelligence, 52(1), 680–680. https://doi.org/10.1007/s10489-021-02538-5
  • Wan, A., Yang, J., Chen, T., Yang, J., Li, K., & Zhou, Q. (2022). Dynamic pollution emission prediction method of a combined heat and power system based on the hybrid CNN-LSTM model and attention mechanism. Environmental Science and Pollution Research, 29(46), 69918–69931. https://doi.org/10.1007/s11356-022-20718-9
  • Wang, D., Han, C., Wang, L., Cai, E., & Zhang, P. (2023a). Surface roughness prediction of large shaft grinding via attentional CNN-LSTM fusing multiple process signals. International Journal of Advanced Manufacturing Technology, 126, 4925–4936. https://doi.org/10.1007/s00170-023-11454-6
  • Wang, T., Wang, X., Jiang, Y., Sun, Z., Liang, Y., Hu, X., & Ruan, J. (2023b). Hybrid machine learning approach for evapotranspiration estimation of fruit tree in agricultural cyber-physical systems. Ieee Transactions. on Cybernetics, 53(9), 5677–5691. https://doi.org/10.1109/TCYB.2022.3164542
  • Wang, Y., Chen, J., Chen, X., Zeng, X., Kong, Y., Sun, S., Guo, Y., & Liu, Y. (2021). Short-Term load forecasting for industrial customers based on TCN-LightGBM. IEEE Transactions on Power Systems, 36(3), 1984–1997. https://doi.org/10.1109/TPWRS.2020.3028133
  • Wen, Y., Zhu, X., Li, D., Zhao, Q., Cheng, Q., & Peng, Y. (2021). Proteomics-based prognostic signature and nomogram construction of hypoxia microenvironment on deteriorating glioblastoma (GBM) pathogenesis. Scientific Reports, 11. https://doi.org/10.1038/s41598-021-95980-x
  • Xie, X., Wang, B., Wan, T., & Tang, W. (2020). Multivariate abnormal detection for industrial control systems using 1D CNN and GRU. IEEE Access, 8, 88348–88359. https://doi.org/10.1109/ACCESS.2020.2993335
  • Yan, J., Mu, L., Wang, L., Ranjan, R., & Zomaya, A. Y. (2020). Temporal convolutional networks for the advance prediction of ENSO. Scientific Reports, 10. https://doi.org/10.1038/s41598-020-65070-5
  • Yang, G., Du, S., Duan, Q., & Su, J. (2022). Short-term price forecasting method in electricity spot markets based on attention-LSTM-mTCN. Journal of Electrical Engineering & Technology, 17(2), 1009–1018. https://doi.org/10.1007/s42835-021-00973-5
  • Zhang, S., Yu, H., & Zhu, G. (2022). An emotional classification method of Chinese short comment text based on ELECTRA. Connection Science, 34(1), 254–273. https://doi.org/10.1080/09540091.2021.1985968
  • Zhang, S., Zhu, A., Zhu, G., Wei, Z., & Li, G. (2023). Building fake review detection model based on sentiment intensity and PU learning. IEEE Transactions on Neural Networks and Learning Systems, 1–14. https://doi.org/10.1109/tnnls.2023.3234427
  • Zheng, W., Zhang, S., Yang, C., & Hu, P. (2023). Lightweight multilayer interactive attention network for aspect-based sentiment analysis. Connection Science, 35. https://doi.org/10.1080/09540091.2023.2189119