Robust sequential design for piecewise-stationary multi-armed bandit problem in the presence of outliers

Figures & data

Figure 1. Expected rewards for arms in the simulation experiment 1.

Figure 2. Expected cumulative regrets for the M-UCB and RCD-UCB with different α values in the simulation experiment 1.

Figure 3. Expected rewards for arms in simulation experiment 2.

Figure 4. Expected cumulative regrets for M-UCB and RCD-UCB with different α values in the simulation experiment 2.

Table 1. Average number (AVE) and standard deviation (SD) of detected change points by each algorithm in the simulation experiment 1.

		RCD-UCB
	M-UCB	$\alpha =0.01$	$\alpha =0.025$	$\alpha =0.05$	$\alpha =0.1$
AVE	6.61	4.82	3.62	3.36	2.98
SD	1.54	1.27	1.02	0.77	0.67

Table 2. Average cumulative regrets $\mathcal{R}\left(T\right)$ (AVE), standard deviations (SD) and computational time in seconds (TIME) for different algorithms in the simulation experiment 1.

	UCB1	D-UCB	SW-UCB	M-UCB	RCD-UCB
Average	1005.26	614.17	764.99	599.17	346.14
SD	53.06	30.52	34.43	62.57	80.80
TIME	0.37	0.39	0.69	0.51	1.91

Table 3. Average number (AVE) and standard deviation (SD) of detected change points by each algorithm in the simulation experiment 2.

		RCD-UCB
	M-UCB	$\alpha =0.01$	$\alpha =0.025$	$\alpha =0.05$	$\alpha =0.1$
AVE	14.59	8.76	7.90	3.81	2.92
SD	1.69	1.41	1.35	1.40	1.08

Table 4. Average cumulative regrets $\mathcal{R}\left(T\right)$ (AVE), standard deviations (SD) and average computational time in seconds (TIME) for different algorithms in the simulation experiment 2.

	UCB1	D-UCB	SW-UCB	M-UCB	RCD-UCB
AVE	1533.16	1260.24	1192.15	1117.43	603.26
SD	59.74	23.43	27.04	30.9	48.52
TIME	0.36	0.36	0.67	0.46	1.79

Table 5. Average number (AVE) and standard deviation (SD) of detected change points by each algorithm in the simulation experiment 3.

		RCD-UCB
	M-UCB	$\alpha =0.01$	$\alpha =0.025$	$\alpha =0.05$	$\alpha =0.1$
AVE	3.93	4.27	3.89	3.83	3.81
SD	0.26	0.77	0.31	0.50	0.37

Table 6. Average cumulative regrets $\mathcal{R}\left(T\right)$ (AVE) and standard deviations (SD) by each algorithm in the simulation experiment 3.

					RCD-UCB
	UCB1	D-UCB	SW-UCB	M-UCB	$\alpha =0.01$	$\alpha =0.025$	$\alpha =0.05$	$\alpha =0.1$
AVE	1012.20	363.48	338.03	292.89	317.38	295.82	286.93	294.28
SD	5.17	5.13	8.99	8.80	23.83	13.44	18.36	12.36

Figure 5. Average rewards for arms in the elongation data analysis.

Figure 6. Cumulative regrets for different algorithms in the real data analysis.

Table 7. Cumulative regrets $\mathcal{R}\left(T\right)$ for different algorithms in the elongation data analysis.

UCB1	D-UCB	SW-UCB	M-UCB	RCD-UCB
251800.60	64408.95	65417.44	55658.80	38674.90