ABSTRACT
Multiple comparisons for two or more mean vectors are considered when the dimension of the vectors may exceed the sample size, the design may be unbalanced, populations need not be normal, and the true covariance matrices may be unequal. Pairwise comparisons, including comparisons with a control, and their linear combinations are considered. Under fairly general conditions, the asymptotic multivariate distribution of the vector of test statistics is derived whose quantiles can be used in multiple testing. Simulations are used to show the accuracy of the tests. Real data applications are also demonstrated.
1. Introduction
The objective of this work is to present multiple comparisons for mean vectors in a multi-sample problem where the populations need not necessarily be normal, sample sizes and covariance matrices may be unequal, and the dimension of the vectors may exceed the sample sizes. Precisely, let , , be iid random vectors with , , , where denotes the space of real, symmetric, positive-definite, matrices and denotes the distribution function for ith population.
We are interested to develop multiple comparison procedures (MCP) or, correspondingly, simultaneous confidence intervals (SCI), for difference of mean vectors, by relaxing the usual linear model assumptions, e.g. normality and homoscedasticity. Thus, may be non-normal and may be unequal which, along with also allowed to be unequal (unbalanced design), implies a complete multi-sample Behrens-Fisher problem. Further, we allow p to be large, even . These comparisons are of interest as a first post hoc investigation after a global MANOVA hypothesis of equality of all mean vectors is rejected; see Seber [Citation1] or Johnson and Wichern [Citation2].
The multivariate theory offers a number of solutions to this problem for the classical case, , particularly assuming normality and homoscedasticity. The global MANOVA hypotheses are mostly tested by the likelihood-ratio criterion such as Wikls' Λ and its rejection follows by finding out the mean vectors responsible for the global rejection. It commonly begins with a general strategy for a set of comparisons defined as linear combination, , , where , . A case of particular interest is of pairwise differences themselves which includes all possible differences as well as special cases such as comparisons with a control.
The classical case of such comparisons has been extensively investigated; see e.g. Krishnaiah [Citation3,Citation4], Wijsman [Citation5], Kropf [Citation6], Kropf and Läuter [Citation7], Westfall et al. [Citation8], Läuter et al. [Citation9], Conneely and Boehnke [Citation10], Westfall and Troendle [Citation11], Bretz et al. [Citation12], Dickhaus [Citation13], Goeman and Finos [Citation14], Goeman and Solari [Citation15], Guilbaud [Citation16,Citation17], where Dickhaus [Citation18] is a modern, comprehensive book length reference with exhaustive bibliography.
The classical methods for MCP or SCI do not work when and need to be modified. The recent wave of high-dimensional data has motivated a thorough inquiry into new avenues for simultaneous inference which, already complicated enough as compared to global testing, is further exacerbated by the largeness of dimensionality. Of particular concern are the fields like genetics, microarray, agriculture, fMRI, psychology where analysing umpteen amounts of data has become a norm rather than exception; see e.g. Nichols and Hayasaka [Citation19] and Dickhaus [Citation18].
The multiple comparisons introduced in this paper are applicable for such high-dimensional data which, additionally, do not depend on usual assumptions such as normality and homoscedasticity. In fact, concerning normality, the tests can be used for any distribution with finite fourth moment across p-dimensional vector. A distinguishing feature of the proposed tests is that we exclusively derive asymptotic joint distribution of the entire vector of preliminary tests whose quantiles can be directly used to test any number of comparisons of g man vectors. Under a few, mild assumptions, the asymptotic covariance matrix turns out be of very simple form and particularly sparse, not only making the derivation of the limit distribution convenient but also enhancing the applicability of the proposed tests under fairly general conditions.
We begin in the next section with a concise notational set up, to be used throughout the paper, followed by the main tests and their properties. A simulation based evaluation is given in Section 3 and applications are given in Section 4. Section 5 summarizes the main points.
2. Test statistics and their properties
2.1. Notations and preliminary set up
Let the vectors , , , as defined above, be generated by a probability space (, , ) where the probability measure is indexed with parameter and Θ is the parameter space, not necessarily finite. Then is the data matrix for ith sample and , , with parameter space , where , with using , where ⊕ and ⊗ are the Kronecker sum and Kronecker product, respectively. Let and be the usual unbiased estimators of and with , or, using the ith data matrix, , (1) (1) where is centering matrix, is identity matrix, and a vector of 1 s.
Let be a family of hypotheses, finite or infinite, with card, corresponding to families of distributions with parameter space bifurcated into and , according to being null () or alternative () hypothesis, where , . A (non-randomized) test for each is carried out using a test statistic with its space , which similarly bifurcates the sample space into and , with a binary decision φ: where φ = 1 (0) when is rejected (accepted).
As usual, the power function = α (size) if and (power) if . For a sample , is the p-value of with observed value and critical value . The problem of MCP pertains to simultaneously testing a set of G hypotheses For pairwise comparisons of , we have , , with , and for comparisons with a control, with G=g−1, , assuming, without loss of generality, sample 1 as control. In either case, we essentially deal with a vector of test statistics and corresponding vector of observed p-values, .
With several tests being carried out simultaneously, the most serious issue in multiple testing is to effectively control α, i.e. reduce the chance of false positives (FP). Let be the subset corresponding to the true null hypotheses, , with card, and be the subset for which is rejected. Then = card refers to the set of FPs (rejected true hypotheses or type I errors), so that = card is the index of true positives or TPs (rightly rejected null hypotheses or power of test). We, therefore, are interested to keep ( as small (large) as possible. Several error control procedures can be adopted, subject to research questions. For details, see e.g. Hochberg and Tamhane [Citation20], Bretz et al. [Citation12], Dickhaus [Citation18], Goeman and Solari [Citation15], Hemerik and Goeman [Citation21].
In practice, family-wise error control (in the strong sense), FWEs, is the most desired error control and will be our main target in the sequel. It is the proportion of all FPs, i.e. . The simplest way to control FWEs is through Bonferroni inequality which ensures , where equality holds in most cases since , i.e. each of G tests has chance for FP. It offers an efficient control for small to moderate G but is obviously conservative (or has less power) as G becomes large. An alternative option is the false discovery rate, FDR = with as indicator function; see e.g. Dickhaus [Citation18, Ch. 1].
Among other notations used in the sequel, a vector is a column vector with norm = and a matrix norm is Frobenius . The test statistics are formulated as linear combinations of second-order U-statistics of symmetric (product) kernels, , defined as bilinear forms of independent vectors. With a measurable, possibly degenerate, square-integrable, , function, the set up conforms to a Hilbert space equipped with inner product , so that , with an orthonormal decomposition, is a Hilbert-Schmidt kernel; see van der Vaart [Citation22] or Lee [Citation23]. This helps us study the properties of test statistics under flexible conditions, the subject of next section.
2.2. Test statistics and their properties
For the data set up in Section 2.1, let be the test statistic for a (preliminary) hypothesis with the vector of all hypotheses to be simultaneously tested. Thus, for all pairwise differences, , i<j, with , where (2) (2) is the vector of test statistics, a set of simultaneous tests for , with , . Our strategy begins by defining , a test statistic for , valid for where may be non-normal and may be unequal. The limit of is derived under flexible conditions since the multiple tests heavily rest on the properties of . Using these properties, we derive the joint distribution of to be used for MCP for any G. The most salient feature is that the effect of high-dimensionality, , is taken care of in , so that the limit of is mainly influenced by g or G. Now, to define , consider where (3) (3) are one- and two-sample U-statistics, respectively, with symmetric kernels , , , , , , , . Now under , , so that can be used to test . For scaling and appropriate limit, also consider , , . Note that, ⇒, so that , which is same under and , where . Thus, writing , it follows that [see also Citation24] We thus define the two-sample test statistic for as (4) (4) is location-invariant so that we can assume without loss of generality. is defined in Ahmad [Citation25] as a modification of the Hotelling's two-sample statistic to test for high-dimensional data under non-normality and heteroscedasticity. Recall = where and is pooled estimator of [Citation1, see e.g.]. The modification pertains to removing , which does not exist when , and writing since . Properties of are studied under the following assumptions.
Assumption 2.1
, , , .
Assumption 2.2
As , , .
Assumption 2.3
As , , .
Assumption 2.4
As , , , , k=i or k=j.
The assumptions are stated for g samples for their further use in the sequel. Note that, by Assumption 2.3, . If we let be the eigenvalues of , so that be those of , , then Assumption 2.3 and its consequence uniformly bound the first two moments of . Assumption 2.1 is inevitably needed to compute moments of bilinear forms when normality is relaxed. Assumption 2.4 is only needed for distribution under the alternative.
Assumptions 2.2 and 2.3 are mild and frequently used in high-dimensional testing problems. In particular, Assumption 2.3 holds for many commonly used covariance structures. Consider, e.g. as compound symmetric (CS), with as identity matrix, , a vector of 1s, . Then , r = 1, 2. Note that, unlike common practice in the literature, we need not assume similar bound for higher moments of the eigenvalues of , e.g. which may collapse for many useful structures, including CS. Note also that CS belongs to spiked structures where a few eigenvalues dominate the rest, so that the proposed procedures hold for such structures as well. See also discussion after Assumption 2.6 below.
Under these assumptions, the limit of , for , is given in Ahmad [Citation25]. First, , as . The limit obviously approximates and holds both under and . As under , the kernels of and are degenerate, so that [Citation22] , , where , iid. Then and, by Slutsky's lemma, (5) (5) where the limiting moments, , , approximate the first two moments of , , . Thus . The normal limit follows by an application of Hájek-Šidák Lemma [Citation26, p. 183]. The limit under follows by the projection theory of U-statistics. We estimate by using unbiased, consistent estimators of traces in , i.e. , , , defined as = + , and , where , , . The consistent estimator can replace in . Following theorem summarizes the limit. For proof and an extension to multi-sample case, see Ahmad [Citation25].
Theorem 2.5
For in Equation (Equation4(4) (4) ), under Assumptions 2.1–2.4. The limit remains valid by replacing with its consistent estimator defined above.
A few remarks concerning Theorem 2.5 will help us proceed further. First, the limit of holds for any distribution with finite fourth moment. Second, the composition of in terms of U-statistics helps us relax normality and obtain the limit conveniently as the kernels are simple bilinear forms of independent components. The accuracy of for small or moderate and large p is shown through simulations in Ahmad [Citation25]. This also implies that the dimension p is taken care of in the limit of , so that the extension to multiple comparisons will not be much influenced by p. Finally, as converges to in probability, the limit of mainly follows from . Thus, in extending the limit to , we mainly focus on . For this, note that (6) (6) (7) (7) (8) (8) with for , (see Appendix) where, under , (9) (9) independent of . Now, with , , consider the vector (10) (10) where , , a partitioned matrix with diagonal and off-diagonal blocks = , , i.e. (11) (11) , , is vector of 1s, , is identity matrix, ⊕ is Kronecker sum and in is of order with no zero row if j−i−1=0. A closer look at the structure of reveals several aspects which will simplify the computations that follow. Ignoring for simplicity, and denoting , , we can write (12) (12) For any given i, has same off-diagonal element, , with diagonal elements , where = = , j=i+1. For off-diagonal blocks ,
The off-diagonal elements in are mostly 0 and the number of (rows with) zeros increases with increasing j for every i, making an increasingly sparse matrix. However, the distinct non-zero elements in consist of a much smaller set (13) (13) with cardinality . Thus, for any g, we only need to estimate out of elements in order to estimate . For example, for g = 6, 9, 12, 15, 20 samples, = 21, 66, 78, 120, 210 whereas = 120, 1540, 2211, 5565, 18145, respectively. The consistent estimators of these traces are given before Theorem 2.5. Used as plug-in estimators, they lead to a consistent estimator, , of . A further simplification follows from weak (mostly zero) off-diagonal elements as compared to diagonal ones, so that the following assumption holds trivially.
Assumption 2.6
, .
Although, Assumption 2.6 is kept flexible to adjust many covariance structures, it can be shown that the ratio indeed vanishes for most covariance structures, so that Assumption 2.6 encompasses many practical cases, including trivial ones e.g. ; see also Section 4. For the distribution of , consider the moments of in Equations (Equation6(6) (6) )–(Equation9(9) (9) ). Using the projection theory of U-statistics (Appendix), the projection of can be shown as see [Citation25, Appendix B.2]. As is composed of independent components and holds for any pair , the projection of , hence of , consists of sums of these independent components. Further, with converging to a constant in probability, the limit for follows conveniently by the Cramér-Wold device and Slutsky's lemma [Citation22]. Finally, using the plug-in consistent estimators of the elements of , the limit also extends to . We have the following theorem.
Theorem 2.7
For the limit in Equation (Equation14(14) (14) ) holds under Assumptions 2.1–2.6, as . Further, the limit remains valid by replacing with its consistent estimator defined above.
As mentioned above, the off-diagonal elements in vanish under Assumption 2.6 for most covariance matrices, leaving diagonal. This makes the limit in Theorem 2.7 much easier to prove and simpler to use. In particular, with as the estimator of , as discussed after Equation (Equation5(5) (5) ), we can use the Chi-square limit with with estimated as . Alternatively, the corresponding normal limit may be used. In fact, given the structure of the test statistics, and also because the normal limit follows through Chi-square limit, it has been observed that the Chi-square approximation mostly performs relatively better that the normal limit, and is thus strongly recommended for practical applications.
Note that, Theorem 2.7 implies that the limit also holds for any linear combination , . With , we have, for , (14) (14) so that we can also test any linear combination , particularly including any single , using . The corresponding 100(1 - α)% simultaneous confidence interval (SCI) for follows as (15) (15) where is 100(α/2)% quantile of -distribution. Note that, the observed length of this confidence interval is . By the consistency of (Theorems 2.5-2.7) and the continuous mapping theorem, converges to which, under the assumptions, is a finite value, assuming which holds conveniently.
The comparison of treatments with a control is a special case of all pairwise comparisons presented above. Let Sample 1 be treated as control, and the interest is to test it against all other samples, i.e. , , . The vector of tests is (16) (16) which is the first sub-vector of in Equation (Equation2(2) (2) ). Using the related computations, we get , , the first diagonal block of , so that under the assumptions, and, assuming zero off-diagonals, . The multiple tests and corresponding confidence intervals follows from those given for above, without much changes.
3. Simulations
We do a simulation study to assess the performance of the proposed tests, in terms of their size control and power, and also their robustness to the violation of assumptions. We consider g = 3 and 6 samples and generate p-dimensional iid vectors from normal, uniform and exponential distributions. For g=3, we use , (20, 30, 40), (10, 30, 60) and (50, 75, 100), with , where the last sample size triplet corresponds to large samples and penultimate triplet amounts to very unbalanced design. The other two triplets are used to show the accuracy of the tests for small to moderate sample sizes. We use three covariance structures, Compound Symmetry (CS), Autoregressive of order 1, AR(1), and unstructured (UN), defined, respectively, as , , and with (i=j), (i>j), where is identity matrix and is matrix of 1 s.
To include violation of homoscedasticity assumption, we combine the structures as (CS, AR(1, 0.5), AR(1, 0.7)), (AR(1, 0.5), AR(1, 0.7), UN), where 0.5 and 0.7 are ρ values used. We use for all cases. For g=6, we use , (30, 40, 50, 30, 40, 50), (30, 40, 50, 60, 70, 80), with same covariance matrix combinations as used for g=3, repeated for first three and next three populations. Due to the close similarity of the results, we restrict the presentation of power to (CS, AR, AR) combination for g=3 and to normal and exponential distributions, with first two sample size sextuples, for g=6.
For both size and power, we use . For g=3, we test all (three) pairwise hypotheses , i<j, i,j=1, 2, 3, where for g=6, we do comparisons with (sample 1 as) control, that is, , . Moreover, for power, we add non-centrality parameter, defined as with , to population 1 for both g = 3 and 6. This, for g = 3, affects tests for and , whereas for g = 6 and comparisons with control, it affects all tests. The p-values and power are estimated using the asymptotic distribution in Theorem 2.7, averaged over 1000 simulation runs.
For comparison, we also compute, under the same set up, size and power for the most commonly used multiple test procedure, namely max test, , with Bonferroni error control. We thus compute and use as nominal level to exercise Bonferroni control. Note that, both types of error control pertain to family-wise in the strong sense (FWEs); see Section 1. The estimated quantiles, and power, , are reported in Tables , respectively, for g=3 and 6.
Table 1. Estimated size of pairwise comparisons for g = 3: all distributions.
Table 2. Estimated size of comparisons with a control for g=6: All distributions.
Table 3. Estimated power of pairwise comparisons for g=3: All distributions.
Table 4. Estimated power of comparisons with a control for g=6: All distributions.
We observe an accurate size control by the proposed tests for both 3 and 6 samples, under all covariance structures and for all populations. The accuracy for exponential distribution as a serious non-normal case is particularly noticeable. Likewise is the case for the covariance structures involving CS, being highly spiked covariance matrix, with only two distinct eigenvalues. These results depict strong robustness of the tests against several violations of usual assumptions. Similar situation appears for power which steadily increases not only for increasing sample sizes but also for increasing dimension. Note the power converging quickly to 1 for sample sizes as small as 10 or 20, even for exponential distribution. Due to this, we reduce ϑ values for each p as soon as the power approaches its maximum value. For example, for p=500, power was already observed 1 for , hence not reported. We also note, in comparison, that often moves between being conservative to liberal and looses its stability, although it generally shows nice power.
To conclude, the proposed tests can be generally considered for most of practically used distributions and covariance structures, where the dimension may far exceed the sample size, and for a moderate number of independent samples. Note that, theoretically, the asymptotic covariance matrix of the vector of tests, , holds for any g, hence any G, but a large g is practically a rare phenomenon. In most cases, g is a moderate values like 6 or 7, as compared to p which may run into thousands. In this context, the tests may find applicability in a wide array of practical problems. On the other hand, the largeness of g may, at least in a few special contexts, be of interest and is therefore being considered for a future work. It indeed needs a different sort of asymptotics to allow for simultaneously with .
4. Applications
We apply the proposed procedures to two data sets, heretofore called SRBCT and Species data, with g=4 and 5 samples, respectively. The first data set consists of small, round blue cell tumors (SRBCT) observed over four independent groups, including a normal group, with sizes , , , , with dimension p=2308 gene expressions. The second, species, data set consists of p=809 species counts of macrobenthos, observed from n=101 independent sites in five different regions, with sample sizes , , , , , along a long transact of the Norwegian continental shelf.
We have as complete data matrix with for ith sample, where and p are given above. Both data sets represent unbalanced one-way MANOVA designs with g=4 and 5 independent samples, with dimensions p=2308 and 809, and total sample size and 101, respectively.
We begin by testing global hypotheses, i.e. vs for at least one pair , i, j = , with g=4 and 5, respectively. We use MANOVA test statistic proposed, under identical general conditions as used here, in Ahmad [Citation25]. The observed values of the test statistic, (see Equation 8 in the reference), for SRBCT data are 378.1604 and 45.7850, respectively, for Chi-square and normal approximations, with p-value virtually zero in each case. A detailed analysis of species data is already provided in Ahmad [Citation25, Sec. 5], by which = 180.4 and 40.61 for Chi-square and normal approximations, respectively, with p-values again zero. With global hypotheses strongly rejected, we expect to find vectors responsible for this rejection.
For multiple comparisons, we consider sample 1 as control and compare it with the remaining samples for Species data, i.e. we test , j=2, 3, 4, 5, with , whereas we do all G=6 pairwise comparisons for SRBCT data, i.e. , i,j=1, 2, 3, 4, i<j. The vectors of test statistics for Species and SRBCT data, respectively, are computed as with the corresponding vectors of p-values and . The results indicate all means, statistically, discernably different from each other at any reasonable nominal size. For further assessment, we also compute the matrix (see Equation Equation11(11) (11) ) for the two data sets, respectively, of order and , shown in Equations (Equation17(17) (17) ) and (Equation19(19) (19) ). It may be mentioned that the analysis reported above is based on Chi-square approximation which, as already discussed, has relatively better performance than the normal one, and the ratio in Assumption 2.6 is assumed to vanish, so that are used as diagonal matrices. This can be easily witnessed from the matrices computed for the two data sets. It is clear that ignoring the off-diagonal elements does not cause much loss of information concerning the comparisons.
To expand more on this, and to highlight an additional important property of the proposed tests, is also reported in each case; Equations (Equation18(18) (18) ) and (Equation20(20) (20) ). First, we observe that, estimated is a non-singular matrix, hence can be inverted, something that in fact can be shown for in general. Second, this in turn implies that the tests can be defined as affine-invariant, using . As we have not proved this inverse for the general case explicitly, it is left for a later work. Finally, we notice that the off-diagonal elements virtually vanish in the inverses. Thus, in affine-invariant form, the tests may be used even more safely under Assumption 2.6. (17) (17) (18) (18) (19) (19) (20) (20)
5. Discussion and conclusions
In the context of multi-sample multivariate problem, multiple comparisons of mean vectors with very large dimension, possibly much larger than the number of vectors in any sample, are considered. The case is of frequent interest, for example, as a first post hoc assessment of mean vectors after a global MANOVA hypothesis is rejected. All possible pairwise differences and comparisons with a control are treated. In particular, the joint asymptotic distribution, under , is derived whose tail probabilities can be directly used to carry out the multiple tests. Simulations results are used to show the accuracy of the tests, and a comparison with max test is also given.
Following the objectives of the present work, as stated in Section 1, the proposed tests can be used in applied problems requiring simultaneous inference for two or more large mean vectors which might have been sampled from a non-normal distribution and may have unequal covariance matrices as well as the sample sizes. Whereas the test statistics are asymptotically approximated with Chi-square and Normal distributions, it is observed that the former provides relatively better accuracy than the later and is thus highly recommended for practical use.
Disclosure statement
No potential conflict of interest was reported by the author.
ORCID
M. Rauf Ahmad http://orcid.org/0000-0002-5362-5835
References
- Seber GAF. Multivariate observations. New York (NY): Wiley.
- Johnson RA, DW Wichern. Applied multivariate statistical analysis. 6th ed. Upper Saddle River (NJ): Pearson Education; 2007.
- Krishnaiah PR. On the simultaneous ANOVA and MANOVA tests. Ann Inst Stat Math. 1965;17:35–53. doi: 10.1007/BF02868151
- Krishnaiah PR. Simultaneous test procedures under general MANOVA models. In: Krishnaiah PR, editor. Multivariate analysis. Vol II. New York (NY): Academic Press; 1969. p. 121–143.
- Wijsman RA. Constructing all smallest simultaneous confidence sets in a given class with applications to MANOVA. An Stat. 1979;7(5):1003–1018. doi: 10.1214/aos/1176344784
- Kropf S. Hochdimensionale multivariate Verfahren in der medizinischen Statistik. Aachen: Shaker; 2000.
- Kropf S, Läuter J. Multiple tests for different sets of variables using a data-driven ordering of hypotheses, with an application to gene expression data. Biom J. 2002;44:789–800. doi: 10.1002/1521-4036(200210)44:7<789::AID-BIMJ789>3.0.CO;2-#
- Westfall P, Kropf S, Finos L. Weighted FWE-controlling methods in high-dimensional situations. In: Benjamini Y, Bretz F, Sarakr SK, editors. Recent developments in multiple comparison procedures. Vol. 47, IMS Lecture Notes and Monpgraph Series; 2004. p. 143–154.
- Läuter J, Glimm E, Eszlinger M. Search for relevant sets of variables in a high-dimensional setup keeping the familywise error rate. Statist Neerl. 2005;59(3):298–312. doi: 10.1111/j.1467-9574.2005.00290.x
- Conneely KN, Boehnke M. So many correlated tests, so little time! Rapid adjustment of p values for multiple correlated tests. Amer J Human Genet. 2007;81:1158–1168. doi: 10.1086/522036
- Westfall P, Troendle JF. Multiple testing with minimal assumptions. Biom J. 2008;50:745–755. doi: 10.1002/bimj.200710456
- Bretz F, Hothorn T, Westfall P. Multiple comparisons using R. Boca Raton (FL): CRC Press; 2011.
- Dickhaus T. Simultaneous statistical inference in dynamic factor models. Berlin: Humboldt-Universitätzu; 2012. (Discussion paper, 2012-033.).
- Goeman J, Finos L. The inheritance procedure: Multiple testing of tree-structured hypotheses. Stat App Genet Molec Biol. 2012;11:1–18. doi: 10.1515/1544-6115.1554
- Goeman J, Solari A. Multiple hypothesis testing in genomics. Stat Med. 2014;33:1946–1978. doi: 10.1002/sim.6082
- Guilbaud O. Simultaneous confidence regions for closed tests, including Holms-, Hochberg-, and Hommel-related procedures. Biom J. 2012;54:317–342. doi: 10.1002/bimj.201100123
- Guilbaud O. Sharper Confidence Intervals for Hochberg- and Hommel-Related Multiple Tests Based On an Extended Simes Inequality. Statist Biopharm Res. 2014;6:123–136. doi: 10.1080/19466315.2013.872999
- Dickhaus T. Simultaneous statistical inference: with applications in the life sciences. New York (NY): Springer; 2014.
- Nichols T, Hayasaka S. Controlling the familywise error rate in functional neuroimaging: a comparative review. Stat Methods Med Res. 2003;12:419–446. doi: 10.1191/0962280203sm341ra
- Hochberg Y, Tamhane AC. Multiple comparison procedures. New York (NY): Wiley; 1987.
- Hemerik J, Goeman J. False discovery proportion estimation by permutations: confidence for significance analysis of microarrays. J R Statist Soc B. 2018;80:137–155. doi: 10.1111/rssb.12238
- van der Vaart AW. Asymptotic statistics. Cambridge: Cambridge University Press; 1998.
- Lee AJ. U-statistics: theory and practice. Boca Raton (FL): CRC Press; 1990.
- Ahmad MR. Location-invariant multi-sample U-tests for covariance matrices with large dimension. Scand J Stat. 2017;44:500–523.
- Ahmad MR. A unified approach to testing mean vectors with large dimension. AStA Adv Stat Anal. 2018.
- Jiang J. Large sample techniques for statistics. New York (NY): Springer; 2010.
- Koroljuk VS, Borovskich YV. Theory of U-statistics. Dordrecht: Kluwer Academic Press; 1994.
Appendix. Some basic moments
Consider with symmetric kernel and conditional expectation (projection) , c=1,2 so that , with , i=1,2. For with symmetric kernel with and with , the conditional expectations are , , with corresponding variances , . Here, is used when the arguments are evident from the context. Then, the moments of U-statistics follow as given, e.g., in Koroljuk and Borovskich [Citation27] or van der Vaart [Citation22]; see also Ahmad [Citation25, Appendix A]
Using these notations, , , with , , and . For , , with , , , , , . Now = + , , , , , = , , , , where the remaining covariances vanish by independence.