A Permutation Two One-Sided Tests Procedure to Detect Minimal Fold Changes of Gene Expression Levels

Abstract

Current approaches to identifying differentially expressed genes are based either on the fold changes or on the traditional hypotheses of equality. However, the fold changes do not take into consideration the variation in estimation of the average expression. In addition, the use of fold changes is not in the frame of hypothesis testing, and hence the probability associated with errors for decision making of identification of differentially expressed genes cannot be quantified and evaluated. On the other hand, the traditional hypothesis of equality fails to take into consideration the magnitudes of the biologically meaningful fold changes that truly differentiate the genes between populations. Because of the large number of genes tested and small number of samples available for microarray experiments, the false positive rate for differentially expressed genes is quite high and requires further multiplicity adjustments, or use of an arbitrary cutoff for the p-values. However, all these adjustments do not have any biological justification. Hence, based on the interval hypothesis, Liu and Chow proposed a two one-sided tests procedure by consideration of both the minimal biologically meaningful fold changes and statistical significance simultaneously. To incorporate the correlation structure of expression levels among different genes and possible violation of normality assumption, we propose to apply a permutation method to the two one-sided tests procedure. A simulation study is conducted to empirically compare the type I error rate and power of the procedures based on the traditional hypothesis and the proposed permutation two one-sided tests procedure based on the interval hypothesis under various combinations of fold changes, variability, and sample sizes. Simulation results show that the proposed permutation two one-sided tests procedure based on the interval hypothesis not only can control the type I error rate at the nominal level but also provide adequate power to detect differentially expressed genes. Numerical data from public domains illustrate the proposed methods.

Key Words:

• Fold change
• Interval hypothesis
• Multivariate permutation methods
• Power
• Two one-sided tests procedure
• Type I error

ACKNOWLEDGMENTS

We sincerely thank the two anonymous reviewers and one of the guest editors for their careful, thoughtful, and thorough review and comments, which greatly improved the content and presentation of our work. This research was partially supported by the Taiwan National Science Council Grant NSC95 2118-M-002-007-MY2 to Jen-pei Liu.

The views expressed in this article are personal opinions of the authors and may not represent the position of National Taiwan University and the National Health Research Institutes, Taiwan.

Notes

G = the number of genes; n = number of samples; Methods: 1 = unpaired two-sample t-test; 2 = unpaired two-sample t-test with Bonferroni adjustment; 3 = fixed fold change; 4 = combined fold change; 5 = two one-sided tests procedure; 6 = permutation two one-sided test procedure.

G = the number of genes; n = number of samples; Methods: 1 = unpaired two-sample t-test; 2 = unpaired two-sample t-test with Bonferroni adjustment; 3 = fixed fold change; 4 = combined fold change; 5 = two one-sided tests procedure; 6 = permutation two one-sided test procedure.

Methods: 1 = unpaired two-sample t-test; 2 = unpaired two-sample t-test with Bonferroni adjustment; 3 = fixed fold change; 4 = combined fold change; 5 = two one-sided tests procedure; 6 = permutation two one-sided test procedure.

G = the number of genes; n = number of samples; A = Average; Methods: 1 = unpaired two-sample t-test; 2 = unpaired two-sample t-test with Bonferroni adjustment; 3 = fixed fold change; 4 = combined fold change; 5 = two one-sided tests procedure; 6 = permutation two one-sided test procedure.

TOST=Two one-sided tests procedure; PTOST=permutation two one-sided tests procedure.