Cross-Dimensional Inference of Dependent High-Dimensional Data

Abstract

A growing number of modern scientific problems in areas such as genomics, neurobiology, and spatial epidemiology involve the measurement and analysis of thousands of related features that may be stochastically dependent at arbitrarily strong levels. In this work, we consider the scenario where the features follow a multivariate Normal distribution. We demonstrate that dependence is manifested as random variation shared among features, and that standard methods may yield highly unstable inference due to dependence, even when the dependence is fully parameterized and utilized in the procedure. We propose a “cross-dimensional inference” framework that alleviates the problems due to dependence by modeling and removing the variation shared among features, while also properly regularizing estimation across features. We demonstrate the framework on both simultaneous point estimation and multiple hypothesis testing in scenarios derived from the scientific applications of interest.

KEY WORDS:

• Dependent data
• False discovery rate
• High-dimensional biology
• Multiple hypothesis testing
• Simultaneous inference

Acknowledgments

This research was supported in part by the National Institutes of Health (NIH) grant R01 HG002913.