Individualized Group Learning

Abstract

Many massive data sets are assembled through collections of information of a large number of individuals in a population. The analysis of such data, especially in the aspect of individualized inferences and solutions, has the potential to create significant value for practical applications. Traditionally, inference for an individual in the dataset is either solely relying on the information of the individual or from summarizing the information about the whole population. However, with the availability of big data, we have the opportunity, as well as a unique challenge, to make a more effective individualized inference that takes into consideration of both the population information and the individual discrepancy. To deal with the possible heterogeneity within the population while providing effective and credible inferences for individuals in a dataset, this article develops a new approach called the individualized group learning (iGroup). The iGroup approach uses local nonparametric techniques to generate an individualized group by pooling other entities in the population which share similar characteristics with the target individual, even when individual estimates are biased due to limited number of observations. Three general cases of iGroup are discussed, and their asymptotic performances are investigated. Both theoretical results and empirical simulations reveal that, by applying iGroup, the performance of statistical inference on the individual level are ensured and can be substantially improved from inference based on either solely individual information or entire population information. The method has a broad range of applications. An example in financial statistics is presented.

Keywords:

• Bayesian inference
• Clustering
• Fusion learning
• Individualized Inference
• Kernel smoothing
• Nonparametric
• Similarity meaures

Supplemental Material

The supplemental material contains all the proofs for our theoretical results in in Section 3.

Acknowledgments

The authors wish to thank the editor, associate editor, and two referees for their insightful comments and suggestions.

Additional information

Funding

Chen’s research is supported in part by National Science Foundation grants DMS-1737857, IIS-1741390, CCF-1934924, and DMS-2027855. Xie’s research is supported in part by National Science Foundation grants DMS-1737857, DMS-1812048, DMS-2015373 and DMS-2027855.