A General M-estimation Theory in Semi-Supervised Framework

Abstract

We study a class of general M-estimators in the semi-supervised setting, wherein the data are typically a combination of a relatively small labeled dataset and large amounts of unlabeled data. A new estimator, which efficiently uses the useful information contained in the unlabeled data, is proposed via a projection technique. We prove consistency and asymptotic normality, and provide an inference procedure based on $\mathbb{K}$-fold cross-validation. The optimal weights are derived to balance the contributions of the labeled and unlabeled data. It is shown that the proposed method, by taking advantage of the unlabeled data, produces asymptotically more efficient estimation of the target parameters than the supervised counterpart. Supportive numerical evidence is shown in simulation studies. Applications are illustrated in analysis of the homeless data in Los Angeles. Supplementary materials for this article are available online.

Keywords:

• Projection method
• Semi-supervised inference
• Weighted loss function

Supplementary Materials

The supplementary material contains the proof of the theoretical results, detailed discussions of Remark 3 and Remark 5, Tables 1–13, Figures 1–5 and additional simulation studies.

Acknowledgments

The authors wish to thank the editor, the associate editor, and two reviewers for their insightful comments and constructive suggestions that significantly helped improve the article.

Additional information

Funding

Lin’s work was supported by the Hong Kong Research Grants Council (grant no. 14306219 and 14306620), the National Natural Science Foundation of China (grant no. 11961028) and Direct Grants for Research, The Chinese University of Hong Kong. Zhou’s work was supported by the State Key Program of National Natural Science Foundation of China (71931004) and the National Key R&D Program of China (2021YFA1000100, 2021YFA1000101).