![MathJax Logo](/templates/jsp/_style2/_tandf/pb2/images/math-jax.gif)
ABSTRACT
Multiple comparisons for two or more mean vectors are considered when the dimension of the vectors may exceed the sample size, the design may be unbalanced, populations need not be normal, and the true covariance matrices may be unequal. Pairwise comparisons, including comparisons with a control, and their linear combinations are considered. Under fairly general conditions, the asymptotic multivariate distribution of the vector of test statistics is derived whose quantiles can be used in multiple testing. Simulations are used to show the accuracy of the tests. Real data applications are also demonstrated.
1. Introduction
The objective of this work is to present multiple comparisons for mean vectors in a multi-sample problem where the populations need not necessarily be normal, sample sizes and covariance matrices may be unequal, and the dimension of the vectors may exceed the sample sizes. Precisely, let ,
, be iid random vectors with
,
,
, where
denotes the space of real, symmetric, positive-definite,
matrices and
denotes the distribution function for ith population.
We are interested to develop multiple comparison procedures (MCP) or, correspondingly, simultaneous confidence intervals (SCI), for difference of mean vectors, by relaxing the usual linear model assumptions, e.g. normality and homoscedasticity. Thus, may be non-normal and
may be unequal which, along with
also allowed to be unequal (unbalanced design), implies a complete multi-sample Behrens-Fisher problem. Further, we allow p to be large, even
. These comparisons are of interest as a first post hoc investigation after a global MANOVA hypothesis of equality of all mean vectors is rejected; see Seber [Citation1] or Johnson and Wichern [Citation2].
The multivariate theory offers a number of solutions to this problem for the classical case, , particularly assuming normality and homoscedasticity. The global MANOVA hypotheses are mostly tested by the likelihood-ratio criterion such as Wikls' Λ and its rejection follows by finding out the mean vectors responsible for the global rejection. It commonly begins with a general strategy for a set of comparisons defined as linear combination,
,
, where
,
. A case of particular interest is of pairwise differences
themselves which includes all possible differences as well as special cases such as comparisons with a control.
The classical case of such comparisons has been extensively investigated; see e.g. Krishnaiah [Citation3,Citation4], Wijsman [Citation5], Kropf [Citation6], Kropf and Läuter [Citation7], Westfall et al. [Citation8], Läuter et al. [Citation9], Conneely and Boehnke [Citation10], Westfall and Troendle [Citation11], Bretz et al. [Citation12], Dickhaus [Citation13], Goeman and Finos [Citation14], Goeman and Solari [Citation15], Guilbaud [Citation16,Citation17], where Dickhaus [Citation18] is a modern, comprehensive book length reference with exhaustive bibliography.
The classical methods for MCP or SCI do not work when and need to be modified. The recent wave of high-dimensional data has motivated a thorough inquiry into new avenues for simultaneous inference which, already complicated enough as compared to global testing, is further exacerbated by the largeness of dimensionality. Of particular concern are the fields like genetics, microarray, agriculture, fMRI, psychology where analysing umpteen amounts of data has become a norm rather than exception; see e.g. Nichols and Hayasaka [Citation19] and Dickhaus [Citation18].
The multiple comparisons introduced in this paper are applicable for such high-dimensional data which, additionally, do not depend on usual assumptions such as normality and homoscedasticity. In fact, concerning normality, the tests can be used for any distribution with finite fourth moment across p-dimensional vector. A distinguishing feature of the proposed tests is that we exclusively derive asymptotic joint distribution of the entire vector of preliminary tests whose quantiles can be directly used to test any number of comparisons of g man vectors. Under a few, mild assumptions, the asymptotic covariance matrix turns out be of very simple form and particularly sparse, not only making the derivation of the limit distribution convenient but also enhancing the applicability of the proposed tests under fairly general conditions.
We begin in the next section with a concise notational set up, to be used throughout the paper, followed by the main tests and their properties. A simulation based evaluation is given in Section 3 and applications are given in Section 4. Section 5 summarizes the main points.
2. Test statistics and their properties
2.1. Notations and preliminary set up
Let the vectors ,
,
, as defined above, be generated by a probability space (
,
,
) where the probability measure
is indexed with parameter
and Θ is the parameter space, not necessarily finite. Then
is the data matrix for ith sample and
,
, with parameter space
, where
,
with
using
, where ⊕ and ⊗ are the Kronecker sum and Kronecker product, respectively. Let
and
be the usual unbiased estimators of
and
with
, or, using the ith data matrix,
,
(1)
(1) where
is centering matrix,
is identity matrix,
and
a vector of 1 s.
Let be a family of hypotheses, finite or infinite, with card
, corresponding to families of distributions
with parameter space
bifurcated into
and
, according to
being null (
) or alternative (
) hypothesis, where
,
. A (non-randomized) test for each
is carried out using a test statistic
with its space
, which similarly bifurcates the sample space into
and
, with a binary decision φ:
where φ = 1 (0) when
is rejected (accepted).
As usual, the power function = α (size) if
and
(power) if
. For a sample
,
is the p-value of
with observed value
and critical value
. The problem of MCP pertains to simultaneously testing a set of G hypotheses
For pairwise comparisons of
, we have
,
, with
, and for comparisons with a control,
with G=g−1,
, assuming, without loss of generality, sample 1 as control. In either case, we essentially deal with a vector of test statistics
and corresponding vector of observed p-values,
.
With several tests being carried out simultaneously, the most serious issue in multiple testing is to effectively control α, i.e. reduce the chance of false positives (FP). Let be the subset corresponding to the true null hypotheses,
, with card
, and
be the subset for which
is rejected. Then
= card
refers to the set of FPs (rejected true hypotheses or type I errors), so that
= card
is the index of true positives or TPs (rightly rejected null hypotheses or power of test). We, therefore, are interested to keep
(
as small (large) as possible. Several error control procedures can be adopted, subject to research questions. For details, see e.g. Hochberg and Tamhane [Citation20], Bretz et al. [Citation12], Dickhaus [Citation18], Goeman and Solari [Citation15], Hemerik and Goeman [Citation21].
In practice, family-wise error control (in the strong sense), FWEs, is the most desired error control and will be our main target in the sequel. It is the proportion of all FPs, i.e. . The simplest way to control FWEs is through Bonferroni inequality which ensures
, where equality holds in most cases since
, i.e. each of G tests has
chance for FP. It offers an efficient control for small to moderate G but is obviously conservative (or has less power) as G becomes large. An alternative option is the false discovery rate, FDR =
with
as indicator function; see e.g. Dickhaus [Citation18, Ch. 1].
Among other notations used in the sequel, a vector is a column vector with norm
=
and a matrix norm is Frobenius
. The test statistics are formulated as linear combinations of second-order U-statistics of symmetric (product) kernels,
, defined as bilinear forms of independent vectors. With
a measurable, possibly degenerate, square-integrable,
, function, the set up conforms to a Hilbert space
equipped with inner product
, so that
, with an orthonormal decomposition, is a Hilbert-Schmidt kernel; see van der Vaart [Citation22] or Lee [Citation23]. This helps us study the properties of test statistics under flexible conditions, the subject of next section.
2.2. Test statistics and their properties
For the data set up in Section 2.1, let be the test statistic for a (preliminary) hypothesis
with
the vector of all hypotheses to be simultaneously tested. Thus, for all pairwise differences,
, i<j, with
,
where
(2)
(2) is the vector of test statistics, a set of simultaneous tests for
, with
,
. Our strategy begins by defining
, a test statistic for
, valid for
where
may be non-normal and
may be unequal. The limit of
is derived under flexible conditions since the multiple tests heavily rest on the properties of
. Using these properties, we derive the joint distribution of
to be used for MCP for any G. The most salient feature is that the effect of high-dimensionality,
, is taken care of in
, so that the limit of
is mainly influenced by g or G. Now, to define
, consider
where
(3)
(3) are one- and two-sample U-statistics, respectively, with symmetric kernels
,
,
,
,
,
,
,
. Now
under
,
, so that
can be used to test
. For scaling and appropriate limit, also consider
,
,
. Note that,
⇒
,
so that
, which is same under
and
, where
. Thus, writing
, it follows that [see also Citation24]
We thus define the two-sample test statistic for
as
(4)
(4)
is location-invariant so that we can assume
without loss of generality.
is defined in Ahmad [Citation25] as a modification of the Hotelling's two-sample
statistic to test
for high-dimensional data under non-normality and heteroscedasticity. Recall
=
where
and
is pooled estimator of
[Citation1, see e.g.]. The modification pertains to removing
, which does not exist when
, and writing
since
. Properties of
are studied under the following assumptions.
Assumption 2.1
,
,
,
.
Assumption 2.2
As ,
,
.
Assumption 2.3
As ,
,
.
Assumption 2.4
As ,
,
,
, k=i or k=j.
The assumptions are stated for g samples for their further use in the sequel. Note that, by Assumption 2.3, . If we let
be the eigenvalues of
, so that
be those of
,
, then Assumption 2.3 and its consequence uniformly bound the first two moments of
. Assumption 2.1 is inevitably needed to compute moments of bilinear forms when normality is relaxed. Assumption 2.4 is only needed for distribution under the alternative.
Assumptions 2.2 and 2.3 are mild and frequently used in high-dimensional testing problems. In particular, Assumption 2.3 holds for many commonly used covariance structures. Consider, e.g. as compound symmetric (CS),
with
as identity matrix,
,
a vector of 1s,
. Then
, r = 1, 2. Note that, unlike common practice in the literature, we need not assume similar bound for higher moments of the eigenvalues of
, e.g.
which may collapse for many useful structures, including CS. Note also that CS belongs to spiked structures where a few eigenvalues dominate the rest, so that the proposed procedures hold for such structures as well. See also discussion after Assumption 2.6 below.
Under these assumptions, the limit of , for
, is given in Ahmad [Citation25]. First,
, as
. The limit obviously approximates
and holds both under
and
. As
under
, the kernels of
and
are degenerate, so that [Citation22]
,
, where
, iid. Then
and, by Slutsky's lemma,
(5)
(5) where the limiting moments,
,
, approximate the first two moments of
,
,
. Thus
. The normal limit follows by an application of Hájek-Šidák Lemma [Citation26, p. 183]. The limit under
follows by the projection theory of U-statistics. We estimate
by using unbiased, consistent estimators of traces in
, i.e.
,
,
, defined as
=
+
,
and
, where
,
,
. The consistent estimator
can replace
in
. Following theorem summarizes the limit. For proof and an extension to multi-sample case, see Ahmad [Citation25].
Theorem 2.5
For in Equation (Equation4
(4)
(4) ),
under Assumptions 2.1–2.4. The limit remains valid by replacing
with its consistent estimator defined above.
A few remarks concerning Theorem 2.5 will help us proceed further. First, the limit of holds for any distribution with finite fourth moment. Second, the composition of
in terms of U-statistics helps us relax normality and obtain the limit conveniently as the kernels are simple bilinear forms of independent components. The accuracy of
for small or moderate
and large p is shown through simulations in Ahmad [Citation25]. This also implies that the dimension p is taken care of in the limit of
, so that the extension to multiple comparisons will not be much influenced by p. Finally, as
converges to
in probability, the limit of
mainly follows from
. Thus, in extending the limit to
, we mainly focus on
. For this, note that
(6)
(6)
(7)
(7)
(8)
(8) with
for
,
(see Appendix) where, under
,
(9)
(9) independent of
. Now, with
,
, consider the vector
(10)
(10) where
,
, a partitioned matrix with diagonal and off-diagonal blocks
=
,
, i.e.
(11)
(11)
,
,
is vector of 1s,
,
is identity matrix, ⊕ is Kronecker sum and
in
is of order
with no zero row if j−i−1=0. A closer look at the structure of
reveals several aspects which will simplify the computations that follow. Ignoring
for simplicity, and denoting
,
, we can write
(12)
(12) For any given i,
has same off-diagonal element,
, with diagonal elements
, where
=
=
, j=i+1. For off-diagonal blocks
,
The off-diagonal elements in are mostly 0 and the number of (rows with) zeros increases with increasing j for every i, making
an increasingly sparse matrix. However, the distinct non-zero elements in
consist of a much smaller set
(13)
(13) with cardinality
. Thus, for any g, we only need to estimate
out of
elements in order to estimate
. For example, for g = 6, 9, 12, 15, 20 samples,
= 21, 66, 78, 120, 210 whereas
= 120, 1540, 2211, 5565, 18145, respectively. The consistent estimators of these traces are given before Theorem 2.5. Used as plug-in estimators, they lead to a consistent estimator,
, of
. A further simplification follows from weak (mostly zero) off-diagonal elements as compared to diagonal ones, so that the following assumption holds trivially.
Assumption 2.6
,
.
Although, Assumption 2.6 is kept flexible to adjust many covariance structures, it can be shown that the ratio indeed vanishes for most covariance structures, so that Assumption 2.6 encompasses many practical cases, including trivial ones e.g. ; see also Section 4. For the distribution of
, consider the moments of
in Equations (Equation6
(6)
(6) )–(Equation9
(9)
(9) ). Using the projection theory of U-statistics (Appendix), the projection of
can be shown as
see [Citation25, Appendix B.2]. As
is composed of independent components and holds for any pair
, the projection of
, hence of
, consists of sums of these independent components. Further, with
converging to a constant in probability, the limit for
follows conveniently by the Cramér-Wold device and Slutsky's lemma [Citation22]. Finally, using the plug-in consistent estimators of the elements of
, the limit also extends to
. We have the following theorem.
Theorem 2.7
For the limit in Equation (Equation14
(14)
(14) ) holds under Assumptions 2.1–2.6, as
. Further, the limit remains valid by replacing
with its consistent estimator defined above.
As mentioned above, the off-diagonal elements in vanish under Assumption 2.6 for most covariance matrices, leaving
diagonal. This makes the limit in Theorem 2.7 much easier to prove and simpler to use. In particular, with
as the estimator of
, as discussed after Equation (Equation5
(5)
(5) ), we can use the Chi-square limit with
with
estimated as
. Alternatively, the corresponding normal limit may be used. In fact, given the structure of the test statistics, and also because the normal limit follows through Chi-square limit, it has been observed that the Chi-square approximation mostly performs relatively better that the normal limit, and is thus strongly recommended for practical applications.
Note that, Theorem 2.7 implies that the limit also holds for any linear combination ,
. With
, we have, for
,
(14)
(14) so that we can also test any linear combination
, particularly including any single
, using
. The corresponding 100(1 - α)% simultaneous confidence interval (SCI) for
follows as
(15)
(15) where
is 100(α/2)% quantile of
-distribution. Note that, the observed length of this confidence interval is
. By the consistency of
(Theorems 2.5-2.7) and the continuous mapping theorem,
converges to
which, under the assumptions, is a finite value, assuming
which holds conveniently.
The comparison of treatments with a control is a special case of all pairwise comparisons presented above. Let Sample 1 be treated as control, and the interest is to test it against all other samples, i.e. ,
,
. The vector of tests is
(16)
(16) which is the first sub-vector of
in Equation (Equation2
(2)
(2) ). Using the related computations, we get
,
, the first diagonal block of
, so that under the assumptions,
and, assuming zero off-diagonals,
. The multiple tests and corresponding confidence intervals follows from those given for
above, without much changes.
3. Simulations
We do a simulation study to assess the performance of the proposed tests, in terms of their size control and power, and also their robustness to the violation of assumptions. We consider g = 3 and 6 samples and generate p-dimensional iid vectors from normal, uniform and exponential distributions. For g=3, we use , (20, 30, 40), (10, 30, 60) and (50, 75, 100), with
, where the last sample size triplet corresponds to large samples and penultimate triplet amounts to very unbalanced design. The other two triplets are used to show the accuracy of the tests for small to moderate sample sizes. We use three covariance structures, Compound Symmetry (CS), Autoregressive of order 1, AR(1), and unstructured (UN), defined, respectively, as
,
,
and
with
(i=j),
(i>j), where
is identity matrix and
is matrix of 1 s.
To include violation of homoscedasticity assumption, we combine the structures as (CS, AR(1, 0.5), AR(1, 0.7)), (AR(1, 0.5), AR(1, 0.7), UN), where 0.5 and 0.7 are ρ values used. We use for all cases. For g=6, we use
, (30, 40, 50, 30, 40, 50), (30, 40, 50, 60, 70, 80), with same covariance matrix combinations as used for g=3, repeated for first three and next three populations. Due to the close similarity of the results, we restrict the presentation of power to (CS, AR, AR) combination for g=3 and to normal and exponential distributions, with first two sample size sextuples, for g=6.
For both size and power, we use . For g=3, we test all (three) pairwise hypotheses
, i<j, i,j=1, 2, 3, where for g=6, we do comparisons with (sample 1 as) control, that is,
,
. Moreover, for power, we add non-centrality parameter, defined as
with
, to population 1 for both g = 3 and 6. This, for g = 3, affects tests for
and
, whereas for g = 6 and comparisons with control, it affects all tests. The p-values and power are estimated using the asymptotic distribution in Theorem 2.7, averaged over 1000 simulation runs.
For comparison, we also compute, under the same set up, size and power for the most commonly used multiple test procedure, namely max test, , with Bonferroni error control. We thus compute
and use
as nominal level to exercise Bonferroni control. Note that, both types of error control pertain to family-wise in the strong sense (FWEs); see Section 1. The estimated quantiles,
and power,
, are reported in Tables , respectively, for g=3 and 6.
Table 1. Estimated size of pairwise comparisons for g = 3: all distributions.
Table 2. Estimated size of comparisons with a control for g=6: All distributions.
Table 3. Estimated power of pairwise comparisons for g=3: All distributions.
Table 4. Estimated power of comparisons with a control for g=6: All distributions.
We observe an accurate size control by the proposed tests for both 3 and 6 samples, under all covariance structures and for all populations. The accuracy for exponential distribution as a serious non-normal case is particularly noticeable. Likewise is the case for the covariance structures involving CS, being highly spiked covariance matrix, with only two distinct eigenvalues. These results depict strong robustness of the tests against several violations of usual assumptions. Similar situation appears for power which steadily increases not only for increasing sample sizes but also for increasing dimension. Note the power converging quickly to 1 for sample sizes as small as 10 or 20, even for exponential distribution. Due to this, we reduce ϑ values for each p as soon as the power approaches its maximum value. For example, for p=500, power was already observed 1 for , hence not reported. We also note, in comparison, that
often moves between being conservative to liberal and looses its stability, although it generally shows nice power.
To conclude, the proposed tests can be generally considered for most of practically used distributions and covariance structures, where the dimension may far exceed the sample size, and for a moderate number of independent samples. Note that, theoretically, the asymptotic covariance matrix of the vector of tests, , holds for any g, hence any G, but a large g is practically a rare phenomenon. In most cases, g is a moderate values like
6 or 7, as compared to p which may run into thousands. In this context, the tests may find applicability in a wide array of practical problems. On the other hand, the largeness of g may, at least in a few special contexts, be of interest and is therefore being considered for a future work. It indeed needs a different sort of asymptotics to allow for
simultaneously with
.
4. Applications
We apply the proposed procedures to two data sets, heretofore called SRBCT and Species data, with g=4 and 5 samples, respectively. The first data set consists of small, round blue cell tumors (SRBCT) observed over four independent groups, including a normal group, with sizes ,
,
,
, with dimension p=2308 gene expressions. The second, species, data set consists of p=809 species counts of macrobenthos, observed from n=101 independent sites in five different regions, with sample sizes
,
,
,
,
, along a long transact of the Norwegian continental shelf.
We have as complete data matrix with
for ith sample, where
and p are given above. Both data sets represent unbalanced one-way MANOVA designs with g=4 and 5 independent samples, with dimensions p=2308 and 809, and total sample size
and 101, respectively.
We begin by testing global hypotheses, i.e. vs
for at least one pair
, i, j =
, with g=4 and 5, respectively. We use MANOVA test statistic proposed, under identical general conditions as used here, in Ahmad [Citation25]. The observed values of the test statistic,
(see Equation 8 in the reference), for SRBCT data are 378.1604 and 45.7850, respectively, for Chi-square and normal approximations, with p-value virtually zero in each case. A detailed analysis of species data is already provided in Ahmad [Citation25, Sec. 5], by which
= 180.4 and 40.61 for Chi-square and normal approximations, respectively, with p-values again zero. With global hypotheses strongly rejected, we expect to find vectors responsible for this rejection.
For multiple comparisons, we consider sample 1 as control and compare it with the remaining samples for Species data, i.e. we test , j=2, 3, 4, 5, with
, whereas we do all G=6 pairwise comparisons for SRBCT data, i.e.
, i,j=1, 2, 3, 4, i<j. The vectors of test statistics for Species and SRBCT data, respectively, are computed as
with the corresponding vectors of p-values
and
. The results indicate all means, statistically, discernably different from each other at any reasonable nominal size. For further assessment, we also compute the
matrix (see Equation Equation11
(11)
(11) ) for the two data sets, respectively, of order
and
, shown in Equations (Equation17
(17)
(17) ) and (Equation19
(19)
(19) ). It may be mentioned that the analysis reported above is based on Chi-square approximation which, as already discussed, has relatively better performance than the normal one, and the ratio in Assumption 2.6 is assumed to vanish, so that
are used as diagonal matrices. This can be easily witnessed from the matrices computed for the two data sets. It is clear that ignoring the off-diagonal elements does not cause much loss of information concerning the comparisons.
To expand more on this, and to highlight an additional important property of the proposed tests, is also reported in each case; Equations (Equation18
(18)
(18) ) and (Equation20
(20)
(20) ). First, we observe that, estimated
is a non-singular matrix, hence can be inverted, something that in fact can be shown for
in general. Second, this in turn implies that the tests can be defined as affine-invariant, using
. As we have not proved this inverse for the general case explicitly, it is left for a later work. Finally, we notice that the off-diagonal elements virtually vanish in the inverses. Thus, in affine-invariant form, the tests may be used even more safely under Assumption 2.6.
(17)
(17)
(18)
(18)
(19)
(19)
(20)
(20)
5. Discussion and conclusions
In the context of multi-sample multivariate problem, multiple comparisons of mean vectors with very large dimension, possibly much larger than the number of vectors in any sample, are considered. The case is of frequent interest, for example, as a first post hoc assessment of mean vectors after a global MANOVA hypothesis is rejected. All possible pairwise differences and comparisons with a control are treated. In particular, the joint asymptotic distribution, under , is derived whose tail probabilities can be directly used to carry out the multiple tests. Simulations results are used to show the accuracy of the tests, and a comparison with max test is also given.
Following the objectives of the present work, as stated in Section 1, the proposed tests can be used in applied problems requiring simultaneous inference for two or more large mean vectors which might have been sampled from a non-normal distribution and may have unequal covariance matrices as well as the sample sizes. Whereas the test statistics are asymptotically approximated with Chi-square and Normal distributions, it is observed that the former provides relatively better accuracy than the later and is thus highly recommended for practical use.
Disclosure statement
No potential conflict of interest was reported by the author.
ORCID
M. Rauf Ahmad http://orcid.org/0000-0002-5362-5835
References
- Seber GAF. Multivariate observations. New York (NY): Wiley.
- Johnson RA, DW Wichern. Applied multivariate statistical analysis. 6th ed. Upper Saddle River (NJ): Pearson Education; 2007.
- Krishnaiah PR. On the simultaneous ANOVA and MANOVA tests. Ann Inst Stat Math. 1965;17:35–53. doi: 10.1007/BF02868151
- Krishnaiah PR. Simultaneous test procedures under general MANOVA models. In: Krishnaiah PR, editor. Multivariate analysis. Vol II. New York (NY): Academic Press; 1969. p. 121–143.
- Wijsman RA. Constructing all smallest simultaneous confidence sets in a given class with applications to MANOVA. An Stat. 1979;7(5):1003–1018. doi: 10.1214/aos/1176344784
- Kropf S. Hochdimensionale multivariate Verfahren in der medizinischen Statistik. Aachen: Shaker; 2000.
- Kropf S, Läuter J. Multiple tests for different sets of variables using a data-driven ordering of hypotheses, with an application to gene expression data. Biom J. 2002;44:789–800. doi: 10.1002/1521-4036(200210)44:7<789::AID-BIMJ789>3.0.CO;2-#
- Westfall P, Kropf S, Finos L. Weighted FWE-controlling methods in high-dimensional situations. In: Benjamini Y, Bretz F, Sarakr SK, editors. Recent developments in multiple comparison procedures. Vol. 47, IMS Lecture Notes and Monpgraph Series; 2004. p. 143–154.
- Läuter J, Glimm E, Eszlinger M. Search for relevant sets of variables in a high-dimensional setup keeping the familywise error rate. Statist Neerl. 2005;59(3):298–312. doi: 10.1111/j.1467-9574.2005.00290.x
- Conneely KN, Boehnke M. So many correlated tests, so little time! Rapid adjustment of p values for multiple correlated tests. Amer J Human Genet. 2007;81:1158–1168. doi: 10.1086/522036
- Westfall P, Troendle JF. Multiple testing with minimal assumptions. Biom J. 2008;50:745–755. doi: 10.1002/bimj.200710456
- Bretz F, Hothorn T, Westfall P. Multiple comparisons using R. Boca Raton (FL): CRC Press; 2011.
- Dickhaus T. Simultaneous statistical inference in dynamic factor models. Berlin: Humboldt-Universitätzu; 2012. (Discussion paper, 2012-033.).
- Goeman J, Finos L. The inheritance procedure: Multiple testing of tree-structured hypotheses. Stat App Genet Molec Biol. 2012;11:1–18. doi: 10.1515/1544-6115.1554
- Goeman J, Solari A. Multiple hypothesis testing in genomics. Stat Med. 2014;33:1946–1978. doi: 10.1002/sim.6082
- Guilbaud O. Simultaneous confidence regions for closed tests, including Holms-, Hochberg-, and Hommel-related procedures. Biom J. 2012;54:317–342. doi: 10.1002/bimj.201100123
- Guilbaud O. Sharper Confidence Intervals for Hochberg- and Hommel-Related Multiple Tests Based On an Extended Simes Inequality. Statist Biopharm Res. 2014;6:123–136. doi: 10.1080/19466315.2013.872999
- Dickhaus T. Simultaneous statistical inference: with applications in the life sciences. New York (NY): Springer; 2014.
- Nichols T, Hayasaka S. Controlling the familywise error rate in functional neuroimaging: a comparative review. Stat Methods Med Res. 2003;12:419–446. doi: 10.1191/0962280203sm341ra
- Hochberg Y, Tamhane AC. Multiple comparison procedures. New York (NY): Wiley; 1987.
- Hemerik J, Goeman J. False discovery proportion estimation by permutations: confidence for significance analysis of microarrays. J R Statist Soc B. 2018;80:137–155. doi: 10.1111/rssb.12238
- van der Vaart AW. Asymptotic statistics. Cambridge: Cambridge University Press; 1998.
- Lee AJ. U-statistics: theory and practice. Boca Raton (FL): CRC Press; 1990.
- Ahmad MR. Location-invariant multi-sample U-tests for covariance matrices with large dimension. Scand J Stat. 2017;44:500–523.
- Ahmad MR. A unified approach to testing mean vectors with large dimension. AStA Adv Stat Anal. 2018.
- Jiang J. Large sample techniques for statistics. New York (NY): Springer; 2010.
- Koroljuk VS, Borovskich YV. Theory of U-statistics. Dordrecht: Kluwer Academic Press; 1994.
Appendix. Some basic moments
Consider with symmetric kernel
and conditional expectation (projection)
, c=1,2 so that
,
with
, i=1,2. For
with symmetric kernel
with
and with
, the conditional expectations are
,
,
with corresponding variances
,
. Here,
is used when the arguments are evident from the context. Then, the moments of U-statistics follow as given, e.g., in Koroljuk and Borovskich [Citation27] or van der Vaart [Citation22]; see also Ahmad [Citation25, Appendix A]
Using these notations, ,
, with
,
,
and
. For
,
, with
,
,
,
,
,
. Now
=
+
,
,
,
,
,
=
,
,
,
, where the remaining covariances vanish by independence.