Optimal Strategies for Sequential Validation of Significant Features from High-Dimensional Genomic Data

Abstract

High-dimensional genomic studies play a key role in identifying critical features that are significantly associated with a phenotypic outcome. The two most important examples are the detection of (1) differentially expressed genes from genome-wide gene expression studies and (2) single-nucleotide polymorphisms (SNPs) from genome-wide association studies. Such experiments are often associated with high noise levels, and the validity of statistical conclusions suffers from low sample size compared to large number of features. The corresponding multiple testing problem calls for the identification of optimal strategies for controlling the numbers of false discoveries and false nondiscoveries. In addition, a frequent validation problem is that features identified as important in one study are often less so in another study. Adjustment for multiple testing in both studies separately increases the risk of missing the crucial features even further. These problems can be addressed by sequential validation strategies, where only significant features identified in one study enter as candidates in the next study. The quality associated with different studies, for example, in terms of noise levels, may vary considerably. By performing simulation studies it is possible to demonstrate that the optimal order for this stepwise procedure is to sort experimental studies according to their quality in descending order. The impact of the method for multiple testing adjustment (Bonferroni-Holm, FDR) was also analyzed. Finally, the sequential validation strategy was applied to three large breast cancer studies with gene expression measurements, confirming the crucial impact of the order of the validation steps in a real-world application.

Acknowledgments

Miriam Lohr and Claudia Köllmann contributed equally to this work. The work of Miriam Lohr, Birte Hellwig, and Jörg Rahnenführer has been supported by the German Research Foundation (DFG, grant RA 870/5-1 and RA 870/4-1). Claudia Köllmann and Katja Ickstadt have been supported by the Research Training School on Statistical Modelling of the German Research Foundation. Jörg Rahnenführer and Katja Ickstadt have also been supported by the German Research Foundation within the Collaborative Research Center SFB 876 “Providing Information by Resource-Constrained Analysis,” projects A3 and C4.