Tri-training and MapReduce-based massive data learning

Abstract

Applications to massive data raise two challenges to supervised learning. First, sufficient training examples to ensure the generalization ability become unavailable, since labelling by experts is expensive; second, it is impossible to load massive data into memory, and the response time is unacceptable by a serial mode. In this paper, we gracefully combine semi-supervised learning with parallel computing to meet these two challenges together. In detail, (1) the co-training style of semi-supervised learning named tri-training is exploited and revised in order to perform learning from the labelled and the unlabelled data. In particular, the co-training process is revised by introducing data editing to remove the newly mislabelled data. (2) The learning algorithm for each individual classifier and the data editing are re-formed as the MapReduce parallel pattern. Experiments on University of California, Irvine, Machine Learning Repositoy data sets and the application to CT images detection show improvement in the accuracy and the scalability to massive data.

Keywords:

• semi-supervised learning
• large-scale learning
• tri-training
• MapReduce
• CT diagnosis

Acknowledgements

This work was supported in part by the National Science Foundation of China under the Grant Nos. 60702033, 60832010, and 60772076, the National High-Tec Research and Development Plant of China under the Grant No. 2007AA01Z171, the Heilongjiang Science Foundation Key Project under the No. ZJG0705, the Science Foundation for Distinguished Young Scholars of Heilongjiang Province in China under the Grant No. JC200611.

Notes

^†Present address: China Mobile Research Institute (CMRI), Beijing 100053, PR China. [email protected]

1. Mahout project [online]. http://lucene.apache.org/mahout/.

2. Hadoop, Welcome to hadoop! [online]. http://lucene.apache.org/hadoop/.

3. Here the standard re-training iteration refers to the re-training process in the basic co-training process of tri-training (described in Section 2.1), where the newly labelled candidate training set is directly used to re-train the individual classifier if the sufficient condition (5) is satisfied.