Comment on ‘Inference after covariate-adaptive randomisation: aspects of methodology and theory’

We congratulate Professor Shao on his exciting and thought-provoking paper and appreciate the Editor's invitation to discuss it. This paper provided a comprehensive review of the methodology and theory for statistical inference under covariate-adaptive randomisation. Covariate-adaptive randomisation is widely used in the design stage of clinical trials to balance baseline covariates that are most relevant to the outcomes. Researchers often use linear regression or analysis of covariance (ANCOVA) to analyse the experimental results in the analysis stage. However, the validity of the resulting inferences is not crystal clear because the usual modelling assumptions might not be justified by covariate-adaptive randomisation. It is essential to develop a model-assisted methodology and theory for statistical inference under covariate-adaptive randomisation, allowing the working model to be arbitrarily misspecified. Professor Shao's paper discussed recent developments in this aspect and made recommendations on using valid and efficient inference procedures under covariate-adaptive randomisation.

As pointed out by Professor Shao, Ye, Yi, et al. (2020) proposed a model-assisted regression approach and showed that the resulting regression-adjusted average treatment effect estimator is more efficient than (as least as efficient as) the difference-in-means estimator, without any modelling assumptions on the potential outcomes and covariates. In other words, the model-assisted inference is efficient and robust to model misspecification.

The efficiency gain and robustness of regression adjustment have been widely investigated under simple randomisation. When there are two treatment arms (treatment and control), Yang and Tsiatis (2001) examined three commonly used regression models for estimating the average treatment effect: $\begin{array}{rl}& Y_i\sim \tau A_i,Y_i\sim \tau A_i+X_i^{\mathrm{T}}\text{β},\\ & Y_i\sim \tau A_i+X_i^{\mathrm{T}}\text{β}+A_i(X_i-\overline{X}{)}^{\mathrm{T}}\gamma ,\end{array}$ where $Y_i$ is the observed outcome, $A_i$ is the treatment assignment indicator, $X_i$ is the vector of covariates, and $\overline{X}=\left(1/n\right)\sum _{i=1}^n{X}_i$ is the vector of covariate means. They showed that these three ordinary-least-squares (OLS) estimators of τ are consistent for the average treatment effect and the third OLS estimator is the most efficient. In fact, as shown by Wang, Ogburn, et al. (2019), when $P(A_i=1)=1/2$ (equal allocation), the second OLS estimator is as efficient as the third one and the usual OLS variance estimator derived from the second regression is also consistent. Thus, we can use standard statistical software, such as lm in R, to construct confidence intervals or tests.

Recently, the efficiency and robustness of regression adjustment under covariate-adaptive randomisation have witnessed significant advances from different perspectives (Bugni et al., 2018, 2019; Ma et al., 2020b; Wang, Susukida, et al., 2019; Ye & Shao, 2018; Ye, Shao, et al., 2020). Under stratified randomisation and a finite population framework, Liu and Yang (2020) also proposed a regression-adjusted average treatment effect estimator and showed that it is more efficient than (as least as efficient as) the stratified difference-in-means estimator, $\hat{\tau }=\sum _{\mathit{z}}\frac{n\left(\mathit{z}\right)}{n}\left\{{\overline{Y}}_1\right(\mathit{z})-{\overline{Y}}_0(\mathit{z}\left)\right\}.$ In practice when there exist small strata, the stratum-specific regression-adjusted vectors ${\widehat{\text{β}}}_a\left(\mathit{z}\right)$ proposed by Ye, Yi, et al. (2020) might lead to inferior performance due to overfitting (Liu & Yang, 2020). To solve this problem, two independent works Ma et al. (2020b) (for two treatment arms) and Ye, Shao, et al. (2020) (for multiple treatment arms) suggested to use the stratum-common regression-adjusted vectors: $\begin{array}{rl}{\widehat{\text{β}}}_a& ={\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{i}\mathrm{n}}_{\text{β}}\sum _{\mathit{z}}\sum _{i\in \mathit{z}}{A}_i[Y_i-{\overline{Y}}_a\left(\mathit{z}\right)\\ & -\{X_i-{\overline{X}}_a(\mathit{z}){\}}^{\mathrm{T}}\text{β}{]}^2,a=0,1,\end{array}$ where $i\in \mathit{z}$ denotes sample in stratum $\mathit{z}$, and ${\overline{X}}_a\left(\mathit{z}\right)$ is the sample mean of the covariates under treatment arm a within stratum $\mathit{z}$. The resulting regression-adjusted average treatment effect estimator is $\begin{array}{rl}{\hat{\tau }}_{\text{ols}}& =\sum _{\mathit{z}}\frac{n\left(\mathit{z}\right)}{n}\left(\right[{\overline{Y}}_1\left(\mathit{z}\right)-\left\{{\overline{X}}_1\right(\mathit{z})-\overline{X}(\mathit{z}\left){\}}^{\mathrm{T}}{\widehat{\text{β}}}_1\right]\\ & -\left[{\overline{Y}}_0\right(\mathit{z})-\{{\overline{X}}_0\left(\mathit{z}\right)-\overline{X}\left(\mathit{z}\right){\}}^{\mathrm{T}}{\widehat{\text{β}}}_0\left]\right),\end{array}$ where $\overline{X}\left(\mathit{z}\right)$ is the sample mean of the covariates within stratum $\mathit{z}$. Ma et al. (2020b) and Ye, Shao, et al. (2020) answered a question raised in Wang, Ogburn, et al. (2019),

It is an open question, to the best of our knowledge, as to what happens when more variables than the stratification indicators are included in the ANCOVA model under such randomisation schemes, in terms of consistency of the ANCOVA estimator and how to compute its asymptotic variance under arbitrary model misspecification.

As shown by Ma et al. (2020b) and Ye, Shao, et al. (2020), ${\hat{\tau }}_{\text{ols}}$ is the OLS estimator of τ in the regression with treatment-by-covariates interactions, (1) $Y_i\sim \tau A_i+U_i^{\mathrm{T}}\text{β}+A_i\{U_i-\overline{U}(\mathit{z}){\}}^{\mathrm{T}}\gamma ,$(1) where $U_i$ includes all stratum indicators and additional covariates $X_i$. From a practical point of view, researchers often use a simpler regression (often called ANCOVA): (2) $Y_i\sim \tau A_i+U_i^{\mathrm{T}}\text{β}.$(2) Ma et al. (2020b) showed that, for equal allocation, i.e., $P(A_i=1)=1/2$, the OLS estimator derived from regression (2(2) $Y_i\sim \tau A_i+U_i^{\mathrm{T}}\text{β}.$(2) ) is as efficient as (asymptotically) that obtained from regression (1(1) $Y_i\sim \tau A_i+U_i^{\mathrm{T}}\text{β}+A_i\{U_i-\overline{U}(\mathit{z}){\}}^{\mathrm{T}}\gamma ,$(1) ). More importantly, the OLS variance estimator derived from regression (2(2) $Y_i\sim \tau A_i+U_i^{\mathrm{T}}\text{β}.$(2) ) is also consistent. These properties hold for almost all covariate-adaptive randomisation including minimisation. Therefore, it is valid and efficient to use ANCOVA under covariate-adaptive randomisation with two treatment arms and equal allocation, even if the linear model is misspecified. However, for clinical trials with more than two treatment arms, the equivalence of the point estimators obtained from regressions (1(1) $Y_i\sim \tau A_i+U_i^{\mathrm{T}}\text{β}+A_i\{U_i-\overline{U}(\mathit{z}){\}}^{\mathrm{T}}\gamma ,$(1) ) and (2(2) $Y_i\sim \tau A_i+U_i^{\mathrm{T}}\text{β}.$(2) ) does not hold anymore (Ye, Shao, et al., 2020). As shown by Ye, Shao, et al. (2020), asymptotically, regression (1(1) $Y_i\sim \tau A_i+U_i^{\mathrm{T}}\text{β}+A_i\{U_i-\overline{U}(\mathit{z}){\}}^{\mathrm{T}}\gamma ,$(1) ) is more efficient than (at least as efficient as) regression (2(2) $Y_i\sim \tau A_i+U_i^{\mathrm{T}}\text{β}.$(2) ), and a nonparametric variance estimator should be used for valid inferences. Another contribution of Ye, Shao, et al. (2020) is that they established the joint asymptotic normality of $\left({\overline{Y}}_1\right(\mathit{z}),\dots ,{\overline{Y}}_K(\mathit{z}\left)\right)$, where K is the number of treatment arms. The joint asymptotic normality is useful for testing more general treatment effects.

At the end of the discussion, we would like to point out that regression adjustment for covariate-adaptive randomisation has been extended to high-dimensional settings where the number of covariates is comparable to or even larger than the sample size, see Ma et al. (2020a) for more details.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by National Natural Science Foundation of China [Grant Number 12071242].

Notes on contributors

Hanzhong Liu

Hanzhong Liu, Associate Professor, Center for Statistical Science, Department of Industrial Engineering, Tsinghua University, Beijing, China.