Dynamic Causal Effects Evaluation in A/B Testing with a Reinforcement Learning Framework

Figures & data

Fig. 1 Causal diagram for MDP under settings where treatments depend on current states only. $(S_t,A_t,Y_t)$ represents the state-treatment-outcome triplet. Solid lines represent causal relationships.

Fig. 2 Empirical rejection probabilities of our test and the two-sample t-test with $\alpha (·)={\alpha }_1(·)$ and of the O’Brien and Fleming sequential test developed by Kharitonov et al. (2015). The left panels depicts the empirical Type-I error and the right panels depicts the empirical power. Settings correspond to alternating-time-interval, adaptive and Markov designs, from top to bottom plots.

Fig. 3 (a) Empirical rejection probabilities of the proposed test and the test based on V-learning. (b) Empirical rejection probabilities of the proposed test and the test derived based on analysis of crossover trials. The shaded area corresponds to the interval $[0.05-1.96\text{MCE},0.05+1.96\text{MCE}]$ where MCE denotes the Monte Carlo error $\sqrt{0.05\times 0.95/500}$.

Fig. 4 Our test statistic (the orange line) and the rejection boundary (the blue line) in the A/A (left plot) and A/B (right plot) experiments.

Table 1 Powers of t-test, DML-based test and the proposed test under Examples 1 and 2, with T = 500, $\delta =0.1$. ${\left\{A_t\right\}}_t$ follow iid Bernoulli distribution with success probability 0.5.

		Example 1		Example 2
t-test 0.76	DML-based test 1	Our test 0.98	t-test 0.04	DML-based test 0.06	Our test 0.73

Kharitonov, E., Vorobev, A., Macdonald, C., Serdyukov, P., and Ounis, I. (2015), “Sequential Testing for Early Stopping of Online Experiments,” in Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 473–482. ACM.

 Google Scholar