Testing exchangeability of multivariate distributions

Figures & data

Table 1. P-values of tests for the gene expressions (GE) dataset of Section 4 with p = 3 and n = 24.

		LR	Rank test	Sign test	NN	ID-rank test
Index	Hollander	Bell and Haller	Snijders	Modarres	Modarres	Modarres
Bivariate permutation tests
1-2	0.768	0.078	0.160	0.779	0.112	0.143
1-3	0.287	0.051	0.286	0.835	0.109	0.116
2-3	0.083	0.001	0.024	0.131	0.060	0.024
Multivariate permutation tests
Fisher	0.039	0.001	0.008	0.341	0.010	0.128
Lipták	0.293	0.000	0.008	0.561	0.014	0.591
Tippett	0.022	0.001	0.003	0.054	0.115	0.075
Bivariate permutation tests using Bonferroni correction
	·	⋆	·	·	·	·
Bivariate permutation tests using the approach of Benjamini–Yekutieli
	·	⋆	·	·	·	·

Notes: Bivariate permutation tests and multivariate permutation tests using the approaches of Fisher (F), Lipták (L), or Tippett (T) are presented here. Significance of tests based on Bonferroni correction and B-Y procedure is denoted by star.

Table 2. Simulation A: Empirical rejection frequencies (in $\mathrm{\%}$).

		LR	Rank test	Sign test	NN	ID-rank test
	Hollander	Bell and Haller	Snijders	Modarres	Modarres	Modarres
Contamination proportion 0 (i.e. $H_0^{\ast }$ holds)
Permutation tests of bivariate symmetry (for individual pairs of variables)
1-2	0.055	0.052	0.046	0.042	0.059	0.054
1-3	0.047	0.055	0.044	0.048	0.055	0.050
2-3	0.053	0.051	0.051	0.049	0.063	0.058
Multivariate procedures
Fisher	0.052	0.052	0.051	0.046	0.055	0.052
Lipták	0.048	0.050	0.052	0.049	0.054	0.061
Tippett	0.061	0.053	0.044	0.044	0.055	0.054
B-Y	0.011	0.014	0.010	0.004	0.009	0.012
Bonferroni	0.019	0.016	0.014	0.017	0.020	0.015
Contamination proportion 0.1
Permutation tests of bivariate symmetry (for individual pairs of variables)
1-2	0.336	0.392	0.367	0.331	0.352	0.413
1-3	0.286	0.335	0.350	0.277	0.318	0.344
2-3	0.081	0.110	0.085	0.074	0.073	0.119
Multivariate procedures
Fisher	0.412	0.463	0.428	0.403	0.407	0.446
Lipták	0.388	0.439	0.406	0.369	0.394	0.425
Tippet	0.449	0.501	0.460	0.427	0.445	0.508
B-Y	0.024	0.027	0.021	0.016	0.021	0.026
Bonferroni	0.031	0.029	0.023	0.026	0.031	0.028
Contamination proportion 0.2
Permutation tests of bivariate symmetry (for individual pairs of variables)
1-2	0.774	0.830	0.742	0.728	0.695	0.846
1-3	0.729	0.806	0.714	0.711	0.647	0.813
2-3	0.122	0.163	0.113	0.091	0.100	0.169
Multivariate procedures
Fisher	0.894	0.955	0.911	0.874	0.905	0.938
Lipták	0.825	0.921	0.863	0.856	0.857	0.912
Tippett	0.933	0.992	0.947	0.923	0.949	0.990
B-Y	0.998	0.999	0.934	0.432	0.897	0.776
Bonferroni	1.000	1.000	0.985	0.721	0.991	0.952

Table 3. Simulation B: Empirical rejection frequencies (in $\mathrm{\%}$).

		LR	Rank test	Sign test	NN	ID-rank test
	Hollander	Bell and Haller	Snijders	Modarres	Modarres	Modarres
Contamination proportion 0 (i.e. $H_0^{\ast }$ holds)
Permutation tests of bivariate symmetry (for individual pairs of variables)
1-2	0.053	0.055	0.046	0.057	0.054	0.058
1-3	0.046	0.052	0.051	0.049	0.049	0.050
2-3	0.055	0.044	0.053	0.053	0.052	0.051
Multivariate procedures
Fisher	0.060	0.052	0.051	0.053	0.050	0.057
Lipták	0.053	0.055	0.054	0.060	0.044	0.052
Tippett	0.051	0.048	0.053	0.052	0.052	0.052
B-Y	0.010	0.011	0.008	0.007	0.010	0.011
Bonferroni	0.015	0.019	0.013	0.014	0.016	0.020
Contamination proportion 0.1
Permutation tests of bivariate symmetry (for individual pairs of variables)
1-2	0.270	0.312	0.229	0.226	0.263	0.303
1-3	0.319	0.361	0.268	0.262	0.315	0.364
2-3	0.195	0.234	0.150	0.137	0.169	0.223
Multivariate procedures
Fisher	0.366	0.431	0.365	0.312	0.362	0.411
Lipták	0.327	0.359	0.313	0.290	0.314	0.376
Tippett	0.419	0.472	0.385	0.375	0.389	0.454
B-Y	0.049	0.051	0.046	0.033	0.047	0.052
Bonferroni	0.044	0.046	0.041	0.038	0.045	0.049
Contamination proportion 0.2
Permutation tests of bivariate symmetry (for individual pairs of variables)
1-2	0.708	0.765	0.692	0.678	0.723	0.797
1-3	0.771	0.815	0.734	0.730	0.779	0.832
2-3	0.270	0.310	0.245	0.263	0.276	0.332
Multivariate procedures
Fisher	0.764	0.818	0.713	0.712	0.769	0.826
Lipták	0.713	0.770	0.697	0.671	0.743	0.769
Tippett	0.814	0.855	0.756	0.730	0.813	0.867
B-Y	0.946	0.972	0.928	0.796	0.904	0.897
Bonferroni	0.962	0.980	0.941	0.848	0.925	0.941

Figure 1. Simulation C: empirical rejection rates for the permutation test of bivariate symmetry for variables 1 and 3 (left), and for the multivariate permutation test using the approach of Fisher (middle) and Tippett (right). The curves correspond to (1) Hollander (stars), (2) LR (circles), (3) rank test of Snijders (squares), (4) sign test (triangles), (5) NN test (diamonds), and (6) ID-rank test (plus signs).

Table 4. Simulation D: Empirical rejection frequencies (in $\mathrm{\%}$).

		LR	Rank test	Sign test	NN	ID-rank test
Method	Hollander	Bell and Haller	Snijders	Modarres	Modarres	Modarres
p = 10, m = 0 (i.e. $H_0^{\ast }$ holds)
Fisher	0.03	0.05	0.05	0.03	0.05	0.05
Lipták	0.05	0.04	0.06	0.04	0.05	0.05
Tippett	0.04	0.05	0.05	0.05	0.07	0.06
B-Y	0.00	0.00	0.00	0.00	0.00	0.00
Bonferroni	0.01	0.01	0.00	0.00	0.00	0.01
p = 10, m = 0.1n
Fisher	0.62	0.64	0.59	0.56	0.60	0.62
Lipták	0.58	0.60	0.55	0.53	0.59	0.61
Tippett	0.70	0.67	0.63	0.58	0.66	0.68
B-Y	0.58	0.61	0.53	0.53	0.59	0.61
Bonferroni	0.53	0.55	0.52	0.50	0.54	0.56
p = 10, m = 0.2n
Fisher	0.94	0.95	0.91	0.86	0.92	0.93
Lipták	0.91	0.91	0.87	0.83	0.89	0.91
Tippett	0.96	0.98	0.94	0.88	0.94	0.97
B-Y	0.90	0.91	0.86	0.81	0.86	0.89
Bonferroni	0.87	0.88	0.82	0.77	0.86	0.88
p = 100, m = 0 (i.e. $H_0^{\ast }$ holds)
Fisher	0.04	0.04	0.06	0.04	0.05	0.06
Lipták	0.04	0.03	0.06	0.05	0.04	0.05
Tippett	0.05	0.05	0.08	0.05	0.06	0.07
B-Y	0.00	0.00	0.00	0.00	0.00	0.00
Bonferroni	0.00	0.00	0.00	0.00	0.00	0.00
p = 100, m = 0.1n
Fisher	0.65	0.69	0.59	0.59	0.61	0.64
Lipták	0.61	0.64	0.58	0.55	0.61	0.60
Tippett	0.72	0.67	0.62	0.62	0.68	0.69
B-Y	0.62	0.64	0.56	0.52	0.59	0.65
Bonferroni	0.57	0.60	0.59	0.53	0.56	0.59
p = 100, m = 0.2n
Fisher	0.99	0.99	0.93	0.90	0.94	0.97
Lipták	0.95	0.96	0.91	0.87	0.91	0.93
Tippett	0.98	0.99	0.95	0.90	0.97	0.98
B-Y	0.93	0.92	0.89	0.84	0.90	0.93
Bonferroni	0.91	0.92	0.86	0.81	0.85	0.89

Table 5. Simulation E: Empirical rejection frequencies (in $\mathrm{\%}$).

		LR	Rank test	Sign test	NN	ID-rank test
	Hollander	Bell and Haller	Snijders	Modarres	Modarres	Modarres
Permutation tests of bivariate symmetry (for a given pair of variables)
1-2	0.062	0.065	0.060	0.057	0.072	0.067
1-3	0.060	0.061	0.058	0.061	0.065	0.064
2-3	0.059	0.059	0.064	0.061	0.071	0.071
Multivariate procedures
Fisher	0.125	0.138	0.119	0.106	0.107	0.139
Lipták	0.119	0.132	0.123	0.105	0.102	0.140
Tippett	0.138	0.141	0.131	0.117	0.116	0.144
B-Y	0.023	0.026	0.020	0.009	0.015	0.019
Bonferroni	0.028	0.029	0.025	0.032	0.029	0.024