CFSE: a Chinese short text classification method based on character frequency sub-word enhancement

Figures & data

Figure 1. Model framework.

Figure 2. Process of obtaining CFS corpus.

Figure 3. Structure chart of BiLSTM-Att.

Figure 4. Multi-level mask mechanism.

Figure 5. Structure chart of ERNIE.

Figure 6. Process for obtaining composite features.

Table

Algorithm 1. The composite feature acquisition algorithm.
Input: batch data
Output: composite features
1. C, T = get(batch_data)
2. OC = BiLSTM_Att(C)
3. OT = ERNIE(T)
4. O = Concatenate (OC, OT)
5. Return O
6. End

Figure 7. Diagram of the Softmax classifier.

Figure 8. The network structure of CFSE.

Table

Algorithm 2. CFSE.
Input: Dataset
Output: Classification result
1. Intput(DataSet)
2. CFS_DataSet = CFS(Dataset)
3. DataSet_new = build_dataset (CFS_DataSet, DataSet)
4. model.init(config)
5. for batch_datas in DataSet_new: // model_train
6.  O = model(batch_datas)
7.  labels = Softmax(O)
8.  model.backward()
9.  evaluation(dev_data, model)
10. End for
11. Model = load_model(best_model)
12. Class_result = test(test_data, Model)
13. End

Table 1. Dataset specification.

ID	Dataset	Classes	Total	Train	Dev	Test
1	THUCNews	10	200,000	180,000	10,000	10,000
2	Taobao-PR	2	60,000	50,000	5,000	5,000
3	Sougou-news	10	200,000	180,000	10,000	10,000

Figure 9. Vocab size.

Figure 10. Frequency distribution information of elements.

Table 2. The results of Experiment 2 (THUCNews).

	Precision		Recall		F1-score		Accuracy
	char	CFS	char	CFS	char	CFS	char	CFS
TextCNN	90.04%	89.91%	90.00%	89.87%	89.99%	89.86%	90.00%	89.87%
TextRNN	90.14%	89.49%	89.94%	89.35%	89.98%	89.37%	89.94%	89.35%
TextRNN_Att	90.75%	90.63%	90.61%	90.58%	90.64%	90.58%	90.61%	90.58%
TextRCNN	90.62%	90.57%	90.54%	90.48%	90.53%	90.47%	90.54%	90.48%
DPCNN	90.23%	89.74%	89.82%	89.59%	89.93%	89.60%	89.82%	89.59%
Transformer	88.39%	87.38%	88.05%	86.79%	88.12%	86.95%	88.05%	86.79%
AVG	90.03%	89.62%	89.83%	89.44%	89.87%	89.47%	89.83%	89.44%

Table 3. The results of Experiment 2 (Taobao-PR).

	Precision		Recall		F1-score		Accuracy
	char	CFS	char	CFS	char	CFS	char	CFS
TextCNN	90.21%	89.53%	90.20%	89.52%	90.20%	89.52%	90.20%	89.52%
TextRNN	90.02%	89.41%	89.98%	89.40%	89.98%	89.40%	89.98%	89.40%
TextRNN_Att	90.84%	90.13%	90.84%	90.12%	90.84%	90.12%	90.84%	90.12%
TextRCNN	90.86%	90.59%	90.86%	90.58%	90.86%	90.58%	90.86%	90.58%
DPCNN	89.98%	88.92%	89.90%	88.86%	89.98%	88.85%	89.90%	88.86%
Transformer	88.35%	87.30%	88.34%	89.30%	88.34%	87.30%	88.34%	87.30%
AVG	90.04%	89.31%	90.02%	89.63%	90.03%	89.30%	90.02%	89.30%

Table 4. The results of Experiment 2 (Sougou-news).

	Precision		Recall		F1-score		Accuracy
	char	CFS	char	CFS	char	CFS	char	CFS
TextCNN	86.11%	85.96%	86.07%	85.92%	86.08%	85.92%	86.07%	85.92%
TextRNN	85.90%	85.36%	85.81%	85.30%	85.83%	85.31%	85.81%	85.30%
TextRNN_Att	85.40%	85.34%	85.24%	85.37%	85.25%	85.33%	85.24%	85.37%
TextRCNN	86.20%	86.44%	86.08%	86.22%	86.11%	86.24%	86.08%	86.22%
DPCNN	84.26%	84.05%	84.09%	83.99%	84.12%	83.98%	84.09%	83.99%
Transformer	82.46%	81.15%	82.36%	81.07%	82.33%	81.04%	82.36%	81.07%
AVG	85.06%	84.72%	84.94%	84.65%	84.95%	84.64%	84.94%	84.65%

Table 5. Ablation experiments.

		Mode 1	Mode 2	Mode 3
THUCNews	Precision	90.63%	94.39%	94.97%
	Recall	90.58%	94.32%	94.79%
	F1-score	90.58%	94.34%	94.84%
	Accuracy	90.58%	94.32%	94.83%
Taobao-PR	Precision	90.13%	93.57%	93.74%
	Recall	90.12%	93.56%	93.75%
	F1-score	90.12%	93.56%	93.75%
	Accuracy	90.12%	93.56%	93.75%
Sougou-news	Precision	85.34%	89.85%	90.37%
	Recall	85.37%	89.83%	90.34%
	F1-score	85.33%	89.83%	90.31%
	Accuracy	85.37%	89.83%	90.34%

Table 6. Comparative experiment (THUCNews).

	Precision	Recall	F1-score	Accuracy
TextCNN	90.04%	90.00%	89.99%	90.00%
TextRNN	90.14%	89.94%	89.98%	89.94%
TextRNN_Att	90.75%	90.61%	90.64%	90.61%
TextRCNN	90.62%	90.54%	90.53%	90.54%
DPCNN	90.23%	89.82%	89.93%	89.82%
Transformer	88.39%	88.05%	88.12%	88.05%
BERT	93.99%	93.96%	93.96%	93.96%
CFSE	94.97%	94.79%	94.84%	94.83%

Table 7. Comparative experiment (Taobao-PR).

	Precision	Recall	F1-score	Accuracy
TextCNN	90.21%	90.20%	90.20%	90.20%
TextRNN	90.02%	89.98%	89.98%	89.98%
TextRNN_Att	90.84%	90.84%	90.84%	90.84%
TextRCNN	90.86%	90.86%	90.86%	90.86%
DPCNN	89.98%	89.90%	89.98%	89.90%
Transformer	88.35%	88.34%	88.34%	88.34%
BERT	93.14%	93.14%	93.14%	93.14%
CFSE	93.74%	93.75%	93.75%	93.75%

Table 8. Comparative experiment(Toutiao-news).

	Precision	Recall	F1-score	Accuracy
TextCNN	86.11%	86.07%	86.08%	86.07%
TextRNN	85.90%	85.81%	85.83%	85.81%
TextRNN_Att	85.40%	85.24%	85.25%	85.24%
TextRCNN	86.20%	86.08%	86.11%	86.08%
DPCNN	84.26%	84.09%	84.12%	84.09%
Transformer	82.46%	82.36%	82.33%	82.36%
BERT	89.68%	89.63%	89.62%	89.64%
CFSE	90.37%	90.34%	90.31%	90.34%