A simple but effective method for Indonesian automatic text summarisation

Abstract

Automatic text summarisation (ATS) (therein two main approaches–abstractive summarisation and extractive summarisation are involved) is an automatic procedure for extracting critical information from the text using a specific algorithm or method. Due to the scarcity of corpus, abstractive summarisation achieves poor performance for low-resource language ATS tasks. That’s why it is common for researchers to apply extractive summarisation to low-resource language instead of using abstractive summarisation. As an emerging branch of extraction-based summarisation, methods based on feature analysis quantitate the significance of information by calculating utility scores of each sentence in the article. In this study, we propose a simple but effective extractive method based on the Light Gradient Boosting Machine regression model for Indonesian documents. Four features are extracted, namely PositionScore, TitleScore, the semantic representation similarity between the sentence and the title of document, the semantic representation similarity between the sentence and sentence’s cluster center. We define a formula for calculating the sentence score as the objective function of the linear regression. Considering the characteristics of Indonesian, we use Indonesian lemmatisation technology to improve the calculation of sentence score. The results show that our method is more applicable.

KEYWORDS:

• Automatic text summarisation
• LightGBM
• Indonesian
• regression

1. Introduction

Thanks to the development of Internet technology, we can get access to a vast amount of information easily. However, it also causes such problems as information redundancy, rumours spreading (Zhao et al., 2020), making it hard for people to obtain valuable information from the Internet in a short time (Fang et al., 2020). Hence, it has become an urgent problem to extract effectively the essential and critical knowledge from the Internet.

Artificial intelligence (AI) is a novel technological science used to simulate human intelligence, and its emergence is to change and facilitate people’s lives. As one of the important directions of AI, automatic text summarisation aims at solving “information explosion”, which can automatically generate a shorter version of the source document, retaining its core idea, with some algorithms.

The task has received much attention in natural language processing since it can be used in various fields. Alternatives to achieve Automatic text summarisation (ATS) can be classified into two main approaches, abstractive summarisation and extractive summarisation. One of the critical issues of abstractive summarisation is that it needs a great scale of natural language corpus to train a model. For abstractive summarisation, the ideal summary contains sentences not used in the original text. It usually requires text rewriting to generate a summary, while extractive approaches generate a summary by selecting the most meaningful sentences from the documents. Therefore, the summaries yielded by this kind of method contain some sentences in the source documents without any modification. Apart from the abstractive summarisation and the extractive summarisation, there are many other mature approaches used in universal languages such as English, Chinese, etc. The existing extractive summarisation methods mainly rely on deep learning methods, such as seq2seq. Most of these methods need a great scale of sequence-to-sequence corpus. Owing to the scarcity of corpus, there are few abstractive summarisation approaches used for low-resource language ATS task. Therefore, in this study, we focus on extractive summarisation in the Indonesian ATS task to help Indonesian obtain core information effectively.

A method on the basis of the Light Gradient Boosting Machine (LightGBM Model) (Meng et al., 2016) is proposed here to extract important sentences from Indonesian Documents as summaries, using four contributive features, PositionScore, TitleScore, the semantic representation similarity between the sentence and the title of document (SRST), and the semantic representation similarity between the sentence and sentence’s cluster center (SRSC). Being regarded as a linear regression task in relation to the characteristics of Indonesian, Indonesian ATS is defined as a formula to calculate sentence score as the target function. Moreover, some experiments are conducted to find out the most suitable value of the maximum length of the summary. In these experiments, we not only compared our approach with some advanced ATS models but also compared different regression models with LightGBM.

The main contributions of our research are:

1. A method for the Indonesian automatic text summarisation task is introduced. The experiments conducted reveal that our model could perform what deep Learning methods and LightGBM could for solving extractive summarisation problem in that our model provides a simple but effective way to solve the Indonesian ATS problem.

2. While using the indexes of $F_1$ score of ROUGE-1, $F_1$ score of ROUGE-2 and $F_1$ score of ROUGE-3 to evaluate the performances of the proposed method and other approaches, the result shows that our proposed method performs better on the Indonesian automatic text summarisation task.

3. With the experiment result, we extracted from the characteristics of Indonesian news briefs the fact that Indonesian media prefer using the first and the last sentence of news as the brief.

4. The sentence score is verified by lemmatisation to represent better the correlation between the sentence from the source document and the summarisation.

5. Last but not least, our simple but effective method can be easily and quickly applied to actual industrial scenarios.

2. Previous research

Recently, approaches that use neural network have become a popular trend in extractive summarisation research, which accounts for the majority of the document summaries. Kageback et al. used a recursive autoencoder for the ATS task, achieving the best results on the Opinosis dataset (Kågebäck et al., 2015). On multi-document extractive summarisation task, Yin and Pei (2015) chose the CNN (Convolutional Neural Networks) model to represent the semantic information of the text, using sentences by reducing the cost and considering their “prestige” and “diverseness”. For the multi-document extractive summarisation task, Padmapriya and Duraiswamy (2020) proposed an algorithm to associate different algorithms such as particle swarm optimisation, genetic algorithm, and the deep learning algorithm. Moreover, methods based on feature analysis are also commonly used in the research of extractive summarisations (Osman & Salim, 2020; Patel & Chhinkaniwala, 2018). Text features commonly used include word frequency etc. Baxendale (1995) proposed a feature-based summarisation method in 1995 and Edmundson (1969) added three factors: pragmatic words, title and heading words, and structural indicators to the procedure of extracting features. Kupiec et al. (1995) used a subjective weight combination of features as opposed to training a model to get the weights of features with a corpus. Yu et al. (2016) presented a method based on improved TextRank from PageRank algorithm, to extract features. Scholars also try to use regression algorithms in extractive summarisation. A linear model-based extractive approach by Osborne (2002) considered the connection among various features. A method proposed by Conroy and O’leary (2001) uses a stealth Markov model with some text features to achieve weight computing on sentences. Mani (2016) used MLP and ridge regression to extract summaries, and Sabuna used the decision tree regression method to extract summaries (Sabuna & Setyohadi, 2018). All of them regard extractive summarisation as a logistic regression problem rather than a linear regression problem. For the ATS task, Yadav et al. proposed a hybrid model, which combines statistic theories with semantic theories. This model extracts multiple statistical features, such as sentence position, TF-IDF, etc. (Yadav & Sharan, 2015; Yadav et al., 2016). What is more, they suggested two new approaches used in selecting sentences and proposed the entropy-based summary evaluation criteria in the automatic text document summarisation task (Yadav & Sharan, 2018).

The methods of extractive summarisations are relatively simple to implement, and this kind of solution does not need a large number of resources. Nevertheless, the results incompletely cover the core ideas delivered by the texts. Therefore it is also of great concern that there are redundancies in the summary extracted by these approaches.

The abstractive summarisation ordinarily depends on deep learning methods which have different neural network structure. The well-known Seq2seq model (Cho et al., 2014) is composed of the encoder and the decoder, which are consisted of several layers of RNN and LSTM. Afterwards, Chorowski et al. (2015) introduced the attention mechanism to natural language processing. With the inspiration of Bahdanau’s work, Paulin et al. (2017) initiated using the CNN model in Encoder and Decoder in 2017 and proposed a multi-step attention mechanism. These methods enhanced the performance of the deep learning method. The generative summarisation can convey the main thought of a text without information redundancy. However, this kind of approach requires a major number of corpus for training, and it costs a lot of time in the training process.

At the present, there are more and more researches concentrating on Indonesian ATS task. Fachrurrozi et al. (2013) scored and used the sentences with TF-IDF to reorganise them, and then generated the summaries of news. Silvia et al. (2014) used the LDA method and genetic algorithm to extract summarisations for news texts. Naive Bayes was also used for finishing the task of Indonesian news summarisation (Najibullah, 2015). A study (Gunawan et al., 2017) used the TextTeaser algorithm with some statistical features in extracting news summaries. Slamet et al. (2018) used the TF-IDF method to encode the sentences and scored the similarities against another vector obtained from keywords of the document. These similarity scores were used to obtain sentences as the summary, which was conducive to convey the main thought of the document. Massandy and Khodra (2015) achieved state-of-the-art on TAC 2011 competition. The method’s ROUGE-2 score is approximate to human performance. However, the scale of their dataset with only 56 articles, is limited and is not open to the public. A work done by Fajri Koto (2016) makes it possible for the public to get an Indonesian summarisation dataset. They collected a chat dataset along with two versions of the summaries, which could be considered as a milestone work in Indonesian text summarisation research. Kurniawan and Louvan (2019) also constructed a dataset called INDOSUM and performed some experiments using several models. The result demonstrates that the NEURALSUM (Nallapati et al., 2017) method achieved the best performance.

3. Approach

3.1. Feature extraction

In our feature extraction model, we considered four characteristics of the sentence: position, word similarity between the sentence and the title, semantic similarity between the sentence and the title and semantic similarity to other sentences in the text. With this in mind we designed four-sentence features:

PositionScore is the score calculated according to the sentence’s position, TitleScore is the number of words that appear in the sentence and the title simultaneously, SRST is the semantic representation similarity between the sentence and the title, SRSC, which is the semantic representation similarity between the sentence and the center of an article sentences cluster.

PositionScore: It is generally considered that in news articles, the closer the sentence to the beginning or the end of the news, the higher the importance of the sentence. For this reason, the topic sentence or the sentence at the end of the news gets a higher score than sentences that appear in the middle of the article. The sentence position score named PositionScore is computed by using the following formula: (1) $\text{PositionScore}(o,S)=\frac{o}{(S\cdot 1)}$(1) $o$ is the position of the sentence in the article and $S$ is the total number of sentences. $\text{PositionScore}(o,S)$ is on the basis of Figure 1 which refers to the Textteaser, a summary extraction algorithm (Gunawan et al., 2017). According to the probability distribution of the sentence position in (Gunawan et al., 2017) and the sentence position distribution in Figure 2, we calculated the PositionScore.

Figure 1. Indonesian summarisation extraction.

Figure 2. Sentence position score.

TitleScore: TitleScore is determined by calculating the number of words that appear in the sentence and the title simultaneously since the news titles remain the core information of the news. TitleScore can be calculated by using the below formula. $G$ is the word collection in a sentence $s_i$, and $H$ is the word collection in an article title $t_j$. (2) $\text{TitleScore}(s_i)=\frac{|G\cap H|}{(|t_j|)}$(2) SRST: The TitleScore is only considered by the relationship between title and sentences at the lexical level. We converted sentences and title into vectors by the doc2vec model (Le & Mikolov, 2014), which was trained by ourselves. Doc2Vec algorithm could be exploited to deeply mine and obtain a document’s semantic representation (Wen et al., 2020). We used Antara¹ news to train the doc2vec model. The cosine similarity between vectors of sentences and title can represent the similarities between the title and each sentence at the semantic level. SRST is calculated by using the following formula: (3) $\text{SRST}=\text{cosine}(t,\text{sen})$(3) $t$ represents the text’s title and $\text{sen}$ stands for sentence in the text.

SRSC: In the following experiments, we regarded sentences in one article as a cluster and the distance between each sentence and the cluster center could represent the correlation between the sentence and the topic of the article. The cluster center is the average of all the sentences’ vectors. And we used the same doc2vec model trained in SRST to calculate the cosine similarity. SRSC can be evaluated by using the following formula: (4) $\text{SRSC}=\text{cosine}(c,\text{sen})$(4)

$c$ stands for the cluster center of the sentences and $\text{sen}$ represents sentence in the text.

3.2. Sentence score

The sentence score (defined as the maximum value of the similarity between one sentence and other sentences in the summarisation) represents the correlation between the sentence from the source document and the summarisation. As a mildly agglutinative language, the morphemes within Indonesian words are loosely arranged (Larasati et al., 2011). In Indonesian, two main methods to constitute a new word are affixation and reduplication. Because Indonesian is an agglutinative language, its feature is to generate new words by adding or deleting affixes. For the sentence score to represent the correlation between the sentence from the source document and the summarisation better, we perform lemmatisation to pre-process Indonesian sentences. Generally, lemmatisation is the process of transforming a derivative to its original form based on rules. We choose the Indonesian lemmatisation method, proposed by Lin et al. (2018) to finish the lemmatisation task. We define the formulas (5) and (6) to compute the sentence score. $\text{sen}$ refers to the sentence which needs to be scored and $SS$ is the sentence score of this sentence. $\text{count}\mathrm{\_}\text{relevance}$ is a function to calculate relevance between this sentence $\text{sen}$ and other sentences in the summary. $n$ is the total number of sentences in the summarisation and $s_i$ refers to the sentence in the summarisation. (5) $\begin{array}{rl}& SS=\text{Max}\left({\sum }_i^n\text{count}\mathrm{\_}\text{relevance}(s_i,\text{sen})\right)\end{array}$(5) (6) $\begin{array}{rl}& \text{count}\mathrm{\_}\text{relevance}(s_i,\text{sen})=\frac{|s_i\cup \text{sen}|}{|s_i\cap \text{sen}|}\end{array}$(6) $|s_i\cap \text{sen}|$ represents the union of words in two sentences, $|s_i\cup \text{sen}|$ represents the intersection of words in two sentences.

3.3. Light gradient boosting machine model

To solve the sentence scoring problem, we describe the automatic summarisation task as a regression problem. In this solution, we regard the result $y$ of the regression function $f(x,y)$ as the sentence score which can cover all the features of one sentence. And then, we sort the sentence score in descending order so that we can obtain the most significant sentences with high scores, as text summary.

We used LightGBM model to finish the logistic regression analysis between four features of the sentence and the sentence score. LightGBM, which released by Meng et al. (2016), is a GBDT (Gradient Boosting Decision Tree) algorithm with Gradient based One-Side Sampling (GOSS) and Exclusive Feature Bundling (EFB). GOSS updates the gradients of all samples with large gradients, and operates random sampling on the samples with small gradients. When calculating the information gain, GOSS proposes a constant multiplier for the data instances with small gradients to minimise the impact on the data distribution. GOSS first sorts the data according to the absolute value of the gradient and selects top $a$ instances. Then it randomly samples $b$ instances from the remaining data. Next, when calculating the information gain, GOSS multiplies the sampled small gradient data by $((1-a)/b)$, so that the algorithm will pay more attention to the under-trained instances. EFB regards many exclusive features as one feature to effectively evade needless computation of features whose eigenvalues are zero.

A histogram-based algorithm (Alsabti et al., 1998; Jin & Agrawal, 2003) is used to accelerate and optimise the training process. The algorithm splits the training data into multiple pieces, and each piece of data is used to train a model. Each model conducts local voting to select the top-k attributes. Then the results of all models are integrated and used to select the top-2k attribute. LightGBM finds a leaf with the largest splitter gain by using the leaf-wise strategy.

Given the training set $X=\{(x_i,y_i){\}}_{i=1}^n$, LightGBM takes searching an approximation $\hat{f}(x)$ to a certain function $f^{\ast }(x)$ as the goal. (7) $\hat{f}(x)=\arg \underset{f}{min}{E}_{y,X}L(y,f(x))$(7) The function $\hat{f}(x)$ minimises the value of the loss function $L(y,f(x))$. LightGBM fuses a great deal of $T$ regression trees ${\sum }_{t=1}^T{f}_t(x)$ to make up the final model: (8) $\hat{f}(x)=\sum _{t=1}^T{f}_t(x)$(8) where $T$ is the number of base models. We further determine the base model in the LightGBM model.

The regression trees can be expressed as $w_{q(x),}q\in \{1,2,\dots ,J\}$, where $J$ denotes the total number of leaf nodes, $q$ is the decision rules of the tree and $w$ is the vector representation of the weight of leaf node instances. In step $t$, the model uses the additive form as (9) for training. In LightGBM, it uses Newton’s method to fit the objective function. For simplicity, we removed the constant term in (9) and transformed the formulation as (10).

In formula (10), $g_i$ and $h_i$ denote the first-order and second-order gradient statistics of the loss function. Let $I_j$ represents the sample set of leaf $j$, and (10) could be converted as (11). (9) $\begin{array}{rl}{\Gamma }_t& =\sum _{i=1}^{n}L(y_i,F_{t-1}(x_i)+f_t(x_i))\end{array}$(9) (10) $\begin{array}{rl}{\Gamma }_t& \cong \sum _{i=1}^{n}L\left(g_i{f}_t(x_i)+\frac{1}{2}{h}_i{f}_t^2(x_i)\right)\end{array}$(10) (11) $\begin{array}{rl}{\Gamma }_t& =\sum _{j=1}^{n}L\left(\left(\sum _{i\in I_j}{g}_i\right)w_j+\frac{1}{2}\left(\sum _{i\in I_j}{h}_i+\lambda \right){w_j}^2\right)\end{array}$(11)

For a certain tree structure $q(x)$, each leaf node ${w_j}^{\ast }$’s optimal leaf weight scores and the extreme value of ${\Gamma }_K$ could be calculated as follows: (12) $\begin{array}{rl}{w_j}^{\ast }& =-\frac{{\sum }_{i\in I_j}{g}_i}{{\sum }_{i\in I_j}{h}_i+\lambda }\end{array}$(12) (13) $\begin{array}{rl}{\Gamma }_T^{\ast }& =-\frac{1}{2}\sum _{j=1}^J\frac{{\left({\sum }_{i\in I_j}{g}_i\right)}^2}{{\sum }_{i\in I_j}{h}_i+\lambda }\end{array}$(13) where ${\Gamma }_T^{\ast }$ could be considered as the quality assessment function that evaluated the performance of the tree structure $q$. Finally, the objective function after adding the split is (14) $\text{G}=\frac{1}{2}\left(\frac{{\left({\sum }_{i\in I_L}{g}_i\right)}^2}{{\sum }_{i\in I_L}{h}_i+\lambda }+\frac{{\left({\sum }_{i\in I_R}{g}_i\right)}^2}{{\sum }_{i\in I_R}{h}_i+\lambda }+\frac{{\left({\sum }_{i\in I}{g}_i\right)}^2}{{\sum }_{i\in I}{h}_i+\lambda }\right)$(14) where $I_L$ and $I_R$ are the instance sets of the left and right branches respectively.

3.4. Summary extraction

In the preprocessing process, we split the text sentence by sentence. Considering that Indonesian is an agglutinative language, which creates new words to express different meanings by adding or removing affixes, automatic text summarisation does not depend on features at the affix level. As a result, we decide to process the Indonesian texts in the same way as English texts. With the prediction of the LightGBM model trained in 3.4, all sentences in the article can get their scores that are the symbols of the relevance between sentences and the news topic. The sentence ranking in the summarisation task is very important. Two common methods are based on the original position of the sentence and the sentence score. In this paper, we sort the sentences in the summary according to the scores of the sentences. We sort the sentences in descending order of sentences scores and unify the result in a collection, $sv=\{s_1:v_1,s_2:v_2,\dots ,s_n:v_n\}$. Here $s_i$ refers to a sentence and $v_i$ is the score of the relevance of this sentence. Then we use the method in Table 1 to extract the summary. It selects sentences that obtain the best score among others. As a result, the computation is performed sentence by sentence. While extracting the summary, we suppose a length threshold $\text{thred}$. The threshold $\text{thred}$ is the maximum length of the summarisation, which affects the quality of the summarisation.

Table 1. Abstract extraction pseudo code.

ALGORITHM 1: Abstract extraction

Input: $\text{sv}$

summarization ← “”

for $s_i$ in $\text{sv}$ do

summarization $+=s_i$

if len(summarization) > $\text{thred}$ do

Break

Output: summarization

4. Data and results

In this section, we introduce our implementation, the summarisation datasets and our evaluation indicator, and our results.

4.1. Data

We used the INDOSUM (Kurniawan & Louvan, 2019), a dataset constructed by Kemal Kurniawan and Samuel Louvan, consisting of 18,774 news and summaries. There are title, category, source, URL and an abstractive summary written manually by Indonesian native speakers. We used five-fold cross validation to validate our model and finished five experiments to train and validate our model. During each experiment, we split the dataset into five-folds, then we used four-folds as the train set while the remaining fold was the test set. With the experiment results, we took the average result of the experiments as the final result. Nowadays, there are two main measurement systems (score of ROUGE and $F_1$ score of ROUGE) for evaluating the quality of summaries extracted by algorithms. Although ROUGE is a recall-based metric, there is an option to report precision and $F_1$ scores with each ROUGE metric. The precision score takes the overlapping word units with reference to the word units of the Candidate summary. The $F_1$ score is the harmonic mean of the recall and precision score. In this paper, we used $F_1$ score of ROUGE-1,ROUGE-2,and ROUGE-3 as the evaluation indices.

4.2. Parameters

When extracting the features, we constructed a Doc2vec model to extract the semantic representation of the text. During the training process, we conducted an experiment on a combination of different values for each parameter in the LightGBM model and got the model parameters (as shown in Table 2) under the best performance.

Table 2. The parameters of the model.

Module	Parameter	Value
Doc2vec	min_count	1
	Sample	1e–3
	Negative	5
	Iter	20
LightGBM	boosting_type	gbdt
	max_bin	255
	learning_rate	0.1
	num_leaves	64
	feature_fraction	0.8
	bagging_freq	5
	bagging_fraction	0.8
	min_data_in_leaf	21
	min_sum_hessian_in_leaf	3.0

“max_bin” represents the maximum number of the bin and its default value is 255. LightGBM uses it to compact memory automatically. “learning_rate” stands for the shrinkage step size in the update process. “num_leaves” represents the number of leaf nodes. “feature_fraction” refers to the ratio of randomly selected features to all features in each iteration. “bagging_fraction” refers to choose the part of data randomly with no resampling. “bagging_freq” is the bagging frequency. “min_data_in_leaf” constrains the minimum number of samples for a leaf node. The larger the “min_sum_hessian_in_leaf” parameter, the better the generalisation ability while the smaller the “min_sum_hessian_in_leaf” parameter, the purer the leaf nodes, and the easier it is to overfit.

4.3. Result

In this research, we found that the threshold of the length of the summary, $\text{thred}$, would affect the performance of the model. Therefore we conducted an experiment with different values of the $\text{thred}$ to get the optimal parameter value. Table 3 illustrates that when the $\text{thred}$ took 350, the $F_1$ of ROUGE-1 reached the optimal status and when the $\text{thred}$ took 400, the $F_1$ of ROUGE-2 and the $F_1$ of ROUGE-3 got the highest score. We also analysed the length of the texts in the dataset, and the result shows that the average length of summaries is 422.6 characters. And when we took 400 as the $\text{thred}$, our model achieved the best performance because 400 is the number closest to the average length. With this experiment result, we decide to choose 400 as the optimal value of thred.

Table 3. F₁ SCORE OF ROUGE-1, ROUGE-2, ROUGE-3 of our model with different values of thred.

thred	F1 of ROUGE-1	F1 of ROUGE-2	F1 of ROUGE-3
300	0.737	0.586	0.554
350	0.733	0.599	0.568
400	0.730	0.611	0.579
450	0.709	0.601	0.571

In the experiment, we made the comparison between our approach and five machine learning algorithms: Textteaser (Gunawan et al., 2017), Lead, LexPageRank (Erkan & Radev, 2004), TextRank (Mihalcea & Tarau, 2004) and Submodular (Lin et al., 2018). Textteaser algorithm is implemented by the tool Textteaser.² Others are implemented by the PKUSUMSUM³ tool. At the same time, our method was compared with the two most advanced abstract extraction methods Neural Sum (Najibullah, 2015) and Bert Sum (Liu, 2019), which are based on deep learning. Figure 3 introduces the compare results of all the tested models.

Figure 3. $F_1$ SCORE OF ROUGE-1, ROUGE-2 and ROUGE-3 of all models. *This figure lacks the $F_1$ of ROUGE-3 of Neural Sum because $F_1$ of ROUGE-3 is not adopted in Kurniawan and Louvan, (2019).

We not only compared our method with other summary extraction methods but also compared the LightGBM model we used with other regression models to verify the validity of the GBM model as a regression model in our method. We used SVM, Ridge and MLP (Multi-Layer Perception) as comparison models. The result of the experiment is presented in Table 4.

Table 4. F₁ SCORE OF ROUGE-1, ROUGE-2, ROUGE-3 of all the regression models.

Algorithm	F1 of ROUGE-1	F1 of ROUGE-2	F1 of ROUGE-3
SVM	0.687	0.520	0.487
Ridge	0.677	0.506	0.473
LightGBM	0.730	0.611	0.579
MLP	0.677	0.505	0.472

As experiment results shows adopting the GBM model is better than using other regression models. Moreover, our model achieved a better performance in $F_1$ score of ROUGE-1,ROUGE-3 than other abstract extraction models and only slightly weaker than the neural model in $F_1$ score of ROUGE-2.

We also completed an experiment for evaluating the influence of using lemmatisation in the sentence scoring process. Table 5 illustrates that using lemmatisation can represent the relationship between sentences better and have better performance on the three indicators.

Table 5. Comparison of different sentence scoring methods.

Method	F1 of ROUGE-1	F1 of ROUGE-2	F1 of ROUGE-3
Using lemmatisation	0.730	0.611	0.579
Without using lemmatisation	0.724	0.604	0.574

What’s worth mentioning is that we explored the extent to which different features influence the result of the task in order to understand how Indonesian users write abstracts for news. After removing a feature, we used other features for regression experiments. Table 6 demonstrates that the parameter PositionScore has the greatest impact on Indonesian feature extraction. It suggests that Indonesian users pay more attention to the first and last sentences in the news text when writing news abstracts. Table 6 demonstrates that all the features we extract from the texts are helpful to get better summaries. Both TitleScore and SRST are much more helpful to disclose the relationship between sentence and title at the lexical level and the semantic level.

Table 6. The result of features’ affect experiments.

Unused feature	F1 of ROUGE-1	$\mathrm{\Delta }{F}_1$ of ROUGE-1	F1 of ROUGE-2	$\mathrm{\Delta }{F}_1$ of ROUGE-2	F1 of ROUGE-3	$\mathrm{\Delta }{F}_1$ of ROUGE-3
PositionScore	0.563	−0.167	0.385	−0.226	0.350	−0.229
TitleScore	0.701	−0.019	0.587	−0.024	0.558	−0.021
SRST	0.706	−0.024	0.586	−0.025	0.556	−0.023
SRSC	0.709	−0.011	0.590	−0.021	0.560	−0.019

5. Conclusion

We have proposed here a multi-featured extractive summarisation algorithm based on the LightGBM regression model for Indonesian language. We utilise sentence position information and title information by using the features we extract. In this paper, there are comparisons between the method we propose and other ATS methods. At the same time, the maximum length of the summarisation is adjusted. During the process of the experiment, we find that the optimal value is determined to be 400. In evaluation, our method achieves a better performance in $F_1$ score of ROUGE-1, ROUGE-3 than other abstract extraction models and only slightly weaker than the neural model in $F_1$ score of ROUGE-2. The $F_1$ score of ROUGE-1 of our method reaches 0.730, $F_1$ score of ROUGE-2 achieves 0.611, and $F_1$ score of ROUGE-3 comes up to 0.579. We also explore the extent to which different features affect the task.

The weakness of our method is that the features we use are defined by ourselves. Moreover, the method proposed here only uses the method based on similarity calculation to extract semantic feature. Due to scarcity of the corpus, abstractive summarisation achieves poor performance for the Indonesian automatic text summarisation task. For future research, it is suggested to use deep learning to extract features with the target to extract features automatically. Utilising the deep learning method to extract semantic feature of the text or use a transfer learning model, we’ll try to train with a large scale of English corpora to assist the Indonesian automatic summarisation task.

Disclosure statement

The authors declare that they have no known potential conflicts of interest that may have influenced the work presented in this paper.

Additional information

Funding

This work was supported by Natural Science Foundation of China [Grant Number 61572145]; Guangzhou Science and Technology Plan Project [Grant Number 202009010021]; Major Projects of Guangdong Education Department for Foundation Research and Applied Research [Grant Number 2017KZDXM031].

Notes

1 https://www.antaranews.com/

2 https://github.com/MojoJolo/textteaser

3 https://github.com/PKULCWM/PKUSUMSUM