Markov Neighborhood Regression for High-Dimensional Inference

Abstract

This article proposes an innovative method for constructing confidence intervals and assessing p-values in statistical inference for high-dimensional linear models. The proposed method has successfully broken the high-dimensional inference problem into a series of low-dimensional inference problems: For each regression coefficient β_i, the confidence interval and p-value are computed by regressing on a subset of variables selected according to the conditional independence relations between the corresponding variable X_i and other variables. Since the subset of variables forms a Markov neighborhood of X_i in the Markov network formed by all the variables $X_1,X_2,\dots ,X_p$, the proposed method is coined as Markov neighborhood regression (MNR). The proposed method is tested on high-dimensional linear, logistic, and Cox regression. The numerical results indicate that the proposed method significantly outperforms the existing ones. Based on the MNR, a method of learning causal structures for high-dimensional linear models is proposed and applied to identification of drug sensitive genes and cancer driver genes. The idea of using conditional independence relations for dimension reduction is general and potentially can be extended to other high-dimensional or big data problems as well. Supplementary materials for this article are available online.

Keywords:

• Causal structure discovery
• Confidence interval
• Gaussian graphical model
• p-Value

Supplementary Materials

Supplement description: (i) proofs of Lemma 1, Lemma 2, and Theorem 1, (ii) a variable screening-based MNR method and related theory and numerical results, and (iii) some figures illustrating the performance of MNR in variable selection.

National Institute of General Medical Sciences;

Acknowledgments

The authors thank the editor, associate editor, and two referees for their encouraging and constructive comments which have led to significant improvement of this article.

Additional information

Funding

Liang’s research was partially supported by the NIH grants R01-GM117597 and R01-GM126089.