Comment: inference after covariate-adaptive randomisation: aspects of methodology and theory

We first want to commend (Shao, 2021) for a timely paper that reviews the methodological and theoretical advances in statistical inference after covariate-adaptive randomisation in the last decade. The paper clearly presents the important considerations and pragmatic recommendations when analysing data obtained from covariate-adaptive randomisation, which provides principled guidelines for the practice.

The aim of our remaining comments is to extend the discussion on the invariance property in Shao (2021). That is, the asymptotic distribution of an estimator remains the same under different covariate-adaptive randomisation schemes. For ease of reading, we follow the notation in Shao (2021) whenever possible and focus on the case of two treatment arms (i.e., $k=2$). The ideas can be extended to the case of multiple treatment arms.

For continuous or binary outcomes, Shao (2021) describes three post-stratified estimators for the population mean difference ${\theta }_0=E(Y^{\left(2\right)}-Y^{\left(1\right)})$: $\begin{array}{rl}{\hat{\theta }}_S& =\sum _{\mathbf{z}\in \mathcal{Z}}\frac{n\left(\mathbf{z}\right)}{n}\{{\overline{Y}}_2\left(\mathbf{z}\right)-{\overline{Y}}_1\left(\mathbf{z}\right)\},\\ {\hat{\theta }}_A& =\sum _{\mathbf{z}\in \mathcal{Z}}\frac{n\left(\mathbf{z}\right)}{n}[{\overline{Y}}_2\left(\mathbf{z}\right)-{\overline{Y}}_1\left(\mathbf{z}\right)\\ & -\left\{{\overline{\mathbf{U}}}_2\right(\mathbf{z})-\overline{\mathbf{U}}(\mathbf{z}\left)\right\}{\widehat{\text{β}}}_2\left(\mathbf{z}\right)\\ & +\left\{{\overline{\mathbf{U}}}_1\right(\mathbf{z})-\overline{\mathbf{U}}(\mathbf{z}\left)\right\}{\widehat{\text{β}}}_1\left(\mathbf{z}\right)],\\ {\hat{\theta }}_B& =\sum _{\mathbf{z}\in \mathcal{Z}}\frac{n\left(\mathbf{z}\right)}{n}[{\overline{Y}}_2\left(\mathbf{z}\right)-{\overline{Y}}_1\left(\mathbf{z}\right)\\ & -\left\{{\overline{\mathbf{U}}}_2\right(\mathbf{z})-\overline{\mathbf{U}}(\mathbf{z}\left)\right\}\widehat{\text{β}}\left(\mathbf{z}\right)+\left\{{\overline{\mathbf{U}}}_1\right(\mathbf{z})-\overline{\mathbf{U}}(\mathbf{z}\left)\right\}\widehat{\text{β}}\left(\mathbf{z}\right)],\end{array}$

where $\mathcal{Z}$ is the support of ${\mathbf{Z}}_i$, and all other quantities are defined in Sections 5.2 and 6.1 of Shao (2021). These post-stratified estimators enjoy the invariance property; that is, their asymptotic distributions are not affected by the covariate-adaptive randomisation. This is a very appealing property as (i) the same inference procedure can be universally applied to different covariate-adaptive randomisation schemes; and (ii) valid inference of the treatment effect can be obtained when complicated covariate-adaptive randomisation schemes such as the Pocock and Simon's minimisation are employed.

It is well-known that the post-stratified estimator ${\hat{\theta }}_S$ is algebraically equivalent with the estimator of θ from fitting the following working model $\begin{array}{rl}& E(Y_i\mid A_i,{\mathbf{Z}}_i)\\ & =\alpha +\theta I(A_i=e_2)\\ & +\sum _{t=1}^2\sum _{\mathbf{z}\in {\mathcal{Z}}_{-1}}I(A_i=e_t)\left\{I({\mathbf{Z}}_i=\mathbf{z})-\frac{n\left(\mathbf{z}\right)}{n}\right\}{\eta }_t\left(\mathbf{z}\right),\end{array}$ where $\alpha ,\theta ,{\eta }_1\left(\mathbf{z}\right),{\eta }_2\left(\mathbf{z}\right)$ are unknown parameters, and ${\mathcal{Z}}_{-1}$ is the support of ${\mathbf{Z}}_i$ with one level dropped to avoid degeneracy. See, for example, Fuller (2009, Chapter 2.2.3). Analogously, ${\hat{\theta }}_A$ is algebraically equivalent with the estimator of θ from fitting the following working model $\begin{array}{rl}& E(Y_i\mid A_i,{\mathbf{Z}}_i,{\mathbf{U}}_i)\\ & =\alpha +\theta I(A_i=e_2)\\ & +\sum _{t=1}^2\sum _{\mathbf{z}\in {\mathcal{Z}}_{-1}}I(A_i=e_t)\left\{I({\mathbf{Z}}_i=\mathbf{z})-\frac{n\left(\mathbf{z}\right)}{n}\right\}{\eta }_t\left(\mathbf{z}\right)\\ & +\sum _{t=1}^2\sum _{\mathbf{z}\in \mathcal{Z}}I(A_i=e_t)\\ & \times \left\{I({\mathbf{Z}}_i=\mathbf{z}){\mathbf{U}}_i-\frac{n\left(\mathbf{z}\right)}{n}\overline{\mathbf{U}}\left(\mathbf{z}\right)\right\}{\text{β}}_t\left(\mathbf{z}\right),\end{array}$ and ${\hat{\theta }}_B$ is algebraically equivalent with the estimator of θ from fitting the following working model $\begin{array}{rl}& E(Y_i\mid A_i,{\mathbf{Z}}_i,{\mathbf{U}}_i)\\ & =\alpha +\theta I(A_i=e_2)\\ & +\sum _{t=1}^2\sum _{\mathbf{z}\in {\mathcal{Z}}_{-1}}I(A_i=e_t)\left\{I({\mathbf{Z}}_i=\mathbf{z})-\frac{n\left(\mathbf{z}\right)}{n}\right\}{\eta }_t\left(\mathbf{z}\right)\\ & +\sum _{\mathbf{z}\in \mathcal{Z}}\left\{I({\mathbf{Z}}_i=\mathbf{z}){\mathbf{U}}_i-\frac{n\left(\mathbf{z}\right)}{n}\overline{\mathbf{U}}\left(\mathbf{z}\right)\right\}\text{β}\left(\mathbf{z}\right),\end{array}$ where $\text{β}\left(\mathbf{z}\right),{\text{β}}_1\left(\mathbf{z}\right),{\text{β}}_2\left(\mathbf{z}\right)$ are unknown parameters.

These three invariant estimators ${\hat{\theta }}_S,{\hat{\theta }}_A,{\hat{\theta }}_B$ can each be obtained as the estimator of θ from fitting the following working model, with properly specified ${\mathbf{W}}_i$ and ${\mathbf{V}}_i$ being functions of ${\mathbf{X}}_i$, and $\overline{\mathbf{W}}$ and $\overline{\mathbf{V}}$ being their sample means, (1) $\begin{array}{rl}& E(Y_i\mid A_i,{\mathbf{W}}_i,{\mathbf{V}}_i)\\ & =\alpha +\theta I(A_i=e_2)\\ & +\sum _{t=1}^{2}I(A_i=e_t)({\mathbf{W}}_i-\overline{\mathbf{W}}){\lambda }_{t\mathbf{W}}+({\mathbf{V}}_i-\overline{\mathbf{V}}){\lambda }_{\mathbf{V}},\end{array}$(1) where ${\lambda }_{1\mathbf{W}},{\lambda }_{2\mathbf{W}},{\lambda }_{\mathbf{V}}$ are unknown parameters. Here, the set of covariates included in ${\mathbf{W}}_i$ and ${\mathbf{V}}_i$ are non-overlapping to avoid degeneracy, where ${\mathbf{W}}_i$ has full interactions with $A_i$, while ${\mathbf{V}}_i$ does not have interactions with $A_i$. Specifically, ${\hat{\theta }}_S$ can be obtained with ${\mathbf{W}}_i=\left(I\right({\mathbf{Z}}_i=\mathbf{z}),\mathbf{z}\in {\mathcal{Z}}_{-1}{)}^{\mathrm{T}}$ being a column vector of all dummy variables for ${\mathcal{Z}}_{-1}$ and ${\mathbf{V}}_i$ being empty, ${\hat{\theta }}_A$ can be obtained with ${\mathbf{W}}_i=\left(\right(I({\mathbf{Z}}_i=\mathbf{z}),\mathbf{z}\in {\mathcal{Z}}_{-1}),(I({\mathbf{Z}}_i=\mathbf{z}){\mathbf{U}}_i,\mathbf{z}\in \mathcal{Z}){)}^{\mathrm{T}}$ and ${\mathbf{V}}_i$ being empty, and ${\hat{\theta }}_B$ can be obtained with ${\mathbf{W}}_i=\left(I\right({\mathbf{Z}}_i=\mathbf{z}),\mathbf{z}\in {\mathcal{Z}}_{-1}{)}^{\mathrm{T}}$ and ${\mathbf{V}}_i=\left(I\right({\mathbf{Z}}_i=\mathbf{z}){\mathbf{U}}_i,\mathbf{z}\in \mathcal{Z}{)}^{\mathrm{T}}$.

In fact, model (1(1) $\begin{array}{rl}& E(Y_i\mid A_i,{\mathbf{W}}_i,{\mathbf{V}}_i)\\ & =\alpha +\theta I(A_i=e_2)\\ & +\sum _{t=1}^{2}I(A_i=e_t)({\mathbf{W}}_i-\overline{\mathbf{W}}){\lambda }_{t\mathbf{W}}+({\mathbf{V}}_i-\overline{\mathbf{V}}){\lambda }_{\mathbf{V}},\end{array}$(1) ) defines a general class of estimators of the treatment effect ${\theta }_0$, (2) $\begin{array}{rl}\hat{\theta }& ={\overline{Y}}_2-{\overline{Y}}_1-({\overline{\mathbf{W}}}_2-\overline{\mathbf{W}}){\hat{\lambda }}_{2\mathbf{W}}\\ & +({\overline{\mathbf{W}}}_1-\overline{\mathbf{W}}){\hat{\lambda }}_{1\mathbf{W}}-({\overline{\mathbf{V}}}_2-\overline{\mathbf{V}}){\hat{\lambda }}_{\mathbf{V}}+({\overline{\mathbf{V}}}_1-\overline{\mathbf{V}}){\hat{\lambda }}_{\mathbf{V}},\end{array}$(2) where ${\hat{\lambda }}_{1\mathbf{W}},{\hat{\lambda }}_{2\mathbf{W}},{\hat{\lambda }}_{\mathbf{V}}$ are the least squares estimates from fitting the working model (1(1) $\begin{array}{rl}& E(Y_i\mid A_i,{\mathbf{W}}_i,{\mathbf{V}}_i)\\ & =\alpha +\theta I(A_i=e_2)\\ & +\sum _{t=1}^{2}I(A_i=e_t)({\mathbf{W}}_i-\overline{\mathbf{W}}){\lambda }_{t\mathbf{W}}+({\mathbf{V}}_i-\overline{\mathbf{V}}){\lambda }_{\mathbf{V}},\end{array}$(1) ). Similar to the proof of Theorem 2 in Ye et al. (2020), we can show that the class of estimators defined in (2(2) $\begin{array}{rl}\hat{\theta }& ={\overline{Y}}_2-{\overline{Y}}_1-({\overline{\mathbf{W}}}_2-\overline{\mathbf{W}}){\hat{\lambda }}_{2\mathbf{W}}\\ & +({\overline{\mathbf{W}}}_1-\overline{\mathbf{W}}){\hat{\lambda }}_{1\mathbf{W}}-({\overline{\mathbf{V}}}_2-\overline{\mathbf{V}}){\hat{\lambda }}_{\mathbf{V}}+({\overline{\mathbf{V}}}_1-\overline{\mathbf{V}}){\hat{\lambda }}_{\mathbf{V}},\end{array}$(2) ) are consistent and asymptotically normal and have asymptotic distributions invariant to the covariate-adaptive randomisation schemes, as long as ${\mathbf{W}}_i$ includes the dummy variables for all joint levels of ${\mathbf{Z}}_i$ as a sub-vector. The key step in the proof is to show that for $t=1,2$, (3) $\begin{array}{rl}& E\{Y_i^{\left(t\right)}-E\left(Y_i^{\left(t\right)}\right)-({\mathbf{W}}_i-E({\mathbf{W}}_i\left)\right){\lambda }_{t\mathbf{W}0}\\ & -({\mathbf{V}}_i-E({\mathbf{V}}_i\left)\right){\lambda }_{\mathbf{V}0}\mid {\mathbf{Z}}_i\}=0,\end{array}$(3) where ${\lambda }_{1\mathbf{W}0},{\lambda }_{2\mathbf{W}0},$ and ${\lambda }_{\mathbf{V}0}$ are the probability limits of ${\hat{\lambda }}_{1\mathbf{W}},{\hat{\lambda }}_{2\mathbf{W}},{\hat{\lambda }}_{\mathbf{V}}$ defined as $\begin{array}{rl}& ({\alpha }_0,{\theta }_0,{\lambda }_{\mathbf{V}0},{\lambda }_{1\mathbf{W}0},{\lambda }_{2\mathbf{W}0})\\ & ={\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{i}\mathrm{n}}_{\alpha ,\theta ,{\lambda }_{\mathbf{V}},{\lambda }_{1\mathbf{W}},{\lambda }_{2\mathbf{W}}}E\left[\left\{Y_i-\alpha -\theta I(A_i=e_2)\phantom{\sum _{t=1}^{2^2}}\right.\right.\\ & -\sum _{t=1}^{2}I(A_i=e_t)({\mathbf{W}}_i-E({\mathbf{W}}_i\left)\right){\lambda }_{t\mathbf{W}}\\ & \left.{\left.-({\mathbf{V}}_i-E({\mathbf{V}}_i\left)\right){\lambda }_{\mathbf{V}}\phantom{Y_i-\alpha -\theta I(A_i=e_2)\phantom{\sum _{t=1}^2}}\right\}}^2\right].\end{array}$ Taking the derivatives and rearranging give ${\alpha }_0=E\left(Y^{\left(1\right)}\right)$, ${\theta }_0=E\left(Y^{\left(2\right)}\right)-E\left(Y^{\left(1\right)}\right)$, and $\begin{array}{rl}& E\left[{\mathbf{W}}_i\left\{Y_i^{\left(t\right)}-E\left(Y_i^{\left(t\right)}\right)-({\mathbf{W}}_i-E({\mathbf{W}}_i\left)\right){\lambda }_{t\mathbf{W}0}\right.\right.\\ & \left.\left.-({\mathbf{V}}_i-E({\mathbf{V}}_i\left)\right){\lambda }_{\mathbf{V}0}\phantom{Y_i^{\left(t\right)}-E\left(Y_i^{\left(t\right)}\right)-({\mathbf{W}}_i-E({\mathbf{W}}_i\left)\right){\lambda }_{t\mathbf{W}0}}\right\}\right]=0.\end{array}$ Because ${\mathbf{Z}}_i$ is discrete and ${\mathbf{W}}_i$ contains all joint levels of ${\mathbf{Z}}_i$ as a sub-vector, we have that (3(3) $\begin{array}{rl}& E\{Y_i^{\left(t\right)}-E\left(Y_i^{\left(t\right)}\right)-({\mathbf{W}}_i-E({\mathbf{W}}_i\left)\right){\lambda }_{t\mathbf{W}0}\\ & -({\mathbf{V}}_i-E({\mathbf{V}}_i\left)\right){\lambda }_{\mathbf{V}0}\mid {\mathbf{Z}}_i\}=0,\end{array}$(3) ) holds.

Equation (2(2) $\begin{array}{rl}\hat{\theta }& ={\overline{Y}}_2-{\overline{Y}}_1-({\overline{\mathbf{W}}}_2-\overline{\mathbf{W}}){\hat{\lambda }}_{2\mathbf{W}}\\ & +({\overline{\mathbf{W}}}_1-\overline{\mathbf{W}}){\hat{\lambda }}_{1\mathbf{W}}-({\overline{\mathbf{V}}}_2-\overline{\mathbf{V}}){\hat{\lambda }}_{\mathbf{V}}+({\overline{\mathbf{V}}}_1-\overline{\mathbf{V}}){\hat{\lambda }}_{\mathbf{V}},\end{array}$(2) ) provides a more complete characterisation of the estimators satisfying the invariance property, compared with the ANHECOVA estimator in Ye et al. (2020) derived from (1(1) $\begin{array}{rl}& E(Y_i\mid A_i,{\mathbf{W}}_i,{\mathbf{V}}_i)\\ & =\alpha +\theta I(A_i=e_2)\\ & +\sum _{t=1}^{2}I(A_i=e_t)({\mathbf{W}}_i-\overline{\mathbf{W}}){\lambda }_{t\mathbf{W}}+({\mathbf{V}}_i-\overline{\mathbf{V}}){\lambda }_{\mathbf{V}},\end{array}$(1) ) without the term $({\mathbf{V}}_i-\overline{\mathbf{V}}){\lambda }_{\mathbf{V}}.$ However, we should note that although the class of estimators defined in (2(2) $\begin{array}{rl}\hat{\theta }& ={\overline{Y}}_2-{\overline{Y}}_1-({\overline{\mathbf{W}}}_2-\overline{\mathbf{W}}){\hat{\lambda }}_{2\mathbf{W}}\\ & +({\overline{\mathbf{W}}}_1-\overline{\mathbf{W}}){\hat{\lambda }}_{1\mathbf{W}}-({\overline{\mathbf{V}}}_2-\overline{\mathbf{V}}){\hat{\lambda }}_{\mathbf{V}}+({\overline{\mathbf{V}}}_1-\overline{\mathbf{V}}){\hat{\lambda }}_{\mathbf{V}},\end{array}$(2) ) enjoy the invariance property, they may be less efficient than the simple mean difference ${\overline{Y}}_2-{\overline{Y}}_1$ under some data generating process. This stresses the advantage of the ANHECOVA estimator in Ye et al. (2020) that is guaranteed to be more efficient than the simple mean difference ${\overline{Y}}_2-{\overline{Y}}_1$.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Ting Ye

Ting Ye is a Postdoctoral Fellow at the Department of Statistics, The Wharton School, University of Pennsylvania. She obtained a Ph.D. in Statistics at the University of Wisconsin-Madison in 2019. Her research interest centers around developing pragmatic and robust statistical methods to analyze complex datasets, and advancing causal inference in biomedical and social sciences.

Yanyao Yi

Yanyao Yi is a Research Scientist at the Global Statistical Sciences, Eli Lilly and Company. He obtained a Ph.D. in Statistics at the University of Wisconsin-Madison in 2019. His research mostly focuses on clinical trials.