Laplacian large margin distribution machine for semi-supervised classification

Abstract

Semi-Supervised Learning (SSL) has attracted much attention in the field of machine learning and data mining. As an extension of Support Vector Machine (SVM), the Semi-Supervised Support Vector Machine (S³VM) was proposed for SSL. Recent studies have disclosed that optimising the margin distribution is more crucial than maximising the minimum margin in generating a better classification. However, the existing S³VM models still follow the idea of maximising the minimum margin. Therefore, this paper proposes a novel Laplacian Large margin Distribution Machine (LapLDM) for SSL to enhance the classification performance. This method can optimise the margin distribution by maximising the first-order (margin mean) and minimising the second-order (margin variance) statistics of margins, and exploit the geometry information of marginal distribution embedded in the unlabelled data through the Laplacian regularizer. Then this paper develops a Preconditioned Conjugate Gradient (PCG) algorithm to solve the nonlinear LapLDM model on those regular-scaled data sets and a Stochastic Gradient Descent with Variance Reduction (SVRG) algorithm to solve the linear LapLDM model on those large-scaled data sets. These algorithms can accelerate the implementing efficiencies of proposed models and make them available for those large-scaled problems. Finally, the numerical results on four artificial and fourteen public benchmark data sets demonstrate that the LapLDM is superior to some well-known S³VM models.

Keywords:

• Margin distribution
• Support Vector Machine
• semi-supervised classification
• Laplacian regularizer
• stochastic gradient descent

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 https://www.csie.ntu.edu.tw/cjlin/libsvmtools

2 http://archive.ics.uci.edu/ml/datasets.html

Additional information

Funding

This paper has been supported by the projects funded by the Fundamental Research Funds for the Central Universities #JBK1805005 and the National Natural Science Foundation of China Grant #71701035