Statistical model selection with “Big Data”

Figures & data

Table

K	20	40	100	1000	5000
$N_K$	1750	12300	176,750	167,671,000	$2\times {10}^{10}$
3K	60	120	300	3000	15000

Table 1. Rejection probabilities under the null

Event	Probability	Reject	${\mathsf{p}}_{0.0001}$
$\mathsf{P}\left(|{\mathsf{t}}_i|<c_{\mathit{\alpha }},\forall i=1,\dots N\right)$	${\left(1-\mathit{\alpha }\right)}^N$	0	0.61
$\mathsf{P}\left(|{\mathsf{t}}_i|\ge c_{\mathit{\alpha }}\mid |{\mathsf{t}}_j|<c_{\mathit{\alpha }},\forall j\ne i\right)$	$N\mathit{\alpha }{\left(1-\mathit{\alpha }\right)}^{N-1}$	1	0.30
$\mathsf{P}\left(|{\mathsf{t}}_i|,|{\mathsf{t}}_k|\ge c_{\mathit{\alpha }}\mid |{\mathsf{t}}_j|<c_{\mathit{\alpha }},\forall j\ne i,k\right)$	$\frac{1}{2}N\left(N-1\right){\mathit{\alpha }}^2{\left(1-\mathit{\alpha }\right)}^{N-2}$	2	0.08
$⋮$	$⋮$	$⋮$	$⋮$
$\mathsf{P}\left(|{\mathsf{t}}_i|<c_{\mathit{\alpha }}\mid |{\mathsf{t}}_j|\ge c_{\mathit{\alpha }},\forall i\ne j\right)$	$N{\mathit{\alpha }}^{\left(N-1\right)}\left(1-\mathit{\alpha }\right)$	$N-1$	0
$\mathsf{P}\left(|{\mathsf{t}}_i|\ge c_{\mathit{\alpha }},\forall i=1,\dots N\right)$	${\mathit{\alpha }}^N$	N	0

Table 2. Approximate significance levels and critical values under the null for a normal variable

$\mathit{\alpha }$	0.05	0.01	0.005	0.0025	0.001	0.0001	$2\times {10}^{-7}$
$c_{\mathit{\alpha }}$	1.96	2.575	2.80	3.025	3.30	4.00	6

Table

$t$-test powers
$\mathit{\psi }$	$\mathit{\alpha }$	$c_{\mathit{\alpha }}$	$\mathsf{P}\left(|\mathsf{t}|\ge c_{\mathit{\alpha }}\right)$	${\left[\mathsf{P}\left(|\mathsf{t}|\ge c_{\mathit{\alpha }}\right)\right]}^4$
2	0.05	2.00	0.50	0.063
2	0.01	2.61	0.26	0.005
3	0.0025	3.08	0.50	0.063
3	0.0001	4.00	0.16	0.001
4	0.01	2.61	0.91	0.686
4	0.0001	4.00	0.50	0.063
6	0.001	3.35	0.996	0.984
6	0.0001	4.00	0.976	0.907

Table 3. Correlations of $y_t$ with the $z_{i,t}$

$z_{1,t}$	$z_{2,t}$	$z_{3,t}$	$z_{4,t}$	$z_{5,t}$	$z_{6,t}$	$z_{7,t}$	$z_{8,t}$	$z_{9,t}$	$z_{10,t}$
0.027	–0.029	0.033	–0.038	0.026	–0.030	0.031	–0.035	0.020	–0.037