Questionable Claims for Simple Versions of the Bootstrap

Figures & data

Table 1 Coverage of nominal 95% confidence intervals for for the mean for samples from a normal distribution, using ± twice the bootstrap standard deviation.

n	Coverage
5	84.7%
10	89.3%
30	93.4%
100	96.2%
300	94.8%

Fig. 1 Chi-squared distribution with 0.5 degrees of freedom.

Table 2 Coverage of nominal 95% confidence intervals for the mean using simple bootstrap methods when sampling from a t-distribution with three degrees of freedom.

df	n	Trials	Resamples	Level	Percentile	2SE
3	5	1000	1000	95%	84.7%	87.0%
3	10	1000	1000	95%	90.4%	91.1%
3	30	1000	1000	95%	91.9%	94.7%
3	100	1000	1000	95%	93.7%	94.2%
3	300	1000	1000	95%	95.0%	94.6%

Table 3 Coverage of nominal 90% and 99% confidence intervals for the mean for the percentile bootstrap method when sampling from a t-distribution with three degrees of freedom.

df	n	Trials	Resamples	Coverage90%	Coverage99%
3	5	10,000	10,000	78.4	89.6
3	10	10,000	10,000	83.2	95.6
3	30	10,000	10,000	86.9	97.8
3	100	10,000	10,000	88.2	98.6
3	300	10,000	10,000	89.4	98.9

Table 4 Coverage of nominal 95% confidence intervals for for the mean for simple bootstrap methods when sampling from relatively “tame” distributions.

n	level	2SE cov:t1000	Percentile cov:t1000	Percentile cov:N10k	Percentile cov:1:1000
5	95%	84.8%	83.4%	84.6%	85.2%
10	95%	92.7%	91.3%	91.6%	91.3%
30	95%	94.2%	92.6%	93.5%	93.5%
100	95%	94.9%	93.9%	95.6%	94.6%

Table 5 Coverage of nominal 95% confidence intervals for normal approximation (N) and two simple bootstrap methods when sampling from Binomial(n,p) distributions.

n	Level	Percentile:p = 0.5	2SD:p = 0.5	N:p = 0.5	Percentile:p = 0.9	2SE:p = 0.9	N:p = 0.9
5	95%	93.0%	92.5%	94.5%	39.6%	40.0%	42.2%
10	95%	88.8%	87.1%	89.4%	61.1%	65.9%	67.9%
30	95%	91.6%	94.9%	95.9%	78.6%	87.5%	79.5%
100	95%	94.3%	94.4%	94.4%	91.9%	92.7%	93.5%
300	95%	94.4%	96.0%	95.0%	93.1%	94.5%	93.8%

Table 6 Coverage of nominal 95% confidence intervals for percentile bootstrap method when sampling 30 integers from 1 to 1000.

Parameter name	Parameter value	Coverage
50 percentile (median)	500.5	94.8%
75 percentile (Q3)	750.2	93.8%
90 percentile	900.1	86.7%
95 percentile	950.05	80.2%
99 percentile	990.01	25.5%
100 percentile (maximum)	1000	0.0%
Range	999	0.0%
IQR	499.4	95.8%
Variance	83,416.67	93.1%
SD	288.8194	93.2%
Coeff. of variation	0.5770618	93.8%

Table 7 Coverage of nominal 90% confidence intervals for the median using percentile bootstrap method when sampling from a chi-squared distribution with 0.5 degrees of freedom.

n	Coverage
5	89.6%
10	86.8%
30	87.8%
100	89.5%

Table 8 Coverage of 97.5% one-sided and 95% two-sided confidence intervals for the mean using percentile bootstrap method when sampling from a chi-squared distribution with 0.5 degrees of freedom.

n	Two-sided	Bounded below	Bounded above
5	66.3%	66.1%	98.7%
10	75.7%	75.9%	99.3%
30	85.9%	86.8%	99.1%
100	92.3%	95.0%	99.1%
300	94.3%	96.0%	98.1%

Table 9 Coverage of 95% one and 97.5% two-sided confidence intervals for the mean using ±2SD bootstrap method when sampling from a chi-squared distribution with 0.5 degrees of freedom.

n	Two-sided	Bounded below	Bounded above
5	67.4%	65.9%	99.2%
10	75.0%	74.6%	99.9%
30	87.7%	87.5%	99.9%
100	91.1%	92.9%	99.8%
300	94.1%	94.5%	99.0%

Table 10 Attained alpha of one and two-sided tests for percentile bootstrap method when sampling from a chi-squared distribution with 0.5 degrees of freedom.

n	Two-sided	Bounded below	Bounded above
5	33.7%	33.9%	1.3%
10	24.3%	24.1%	0.7%
30	14.1%	13.2%	0.9%
100	7.7%	5.0%	0.9%
300	5.7%	4.0%	1.9%

Table 11 Attained alpha of one and two-sided tests for ±2SD bootstrap method when sampling from a chi-squared distribution with 0.5 degrees of freedom.

n	Two-sided	Bounded below	Bounded above
5	32.6%	34.1%	0.8%
10	25.0%	25.4%	0.1%
30	12.3%	12.5%	0.1%
100	8.9%	7.1%	0.2%
300	5.9%	5.5%	1.0%