Step-Down Parametric Procedures for Testing Correlated Endpoints in a Group-Sequential Trial

ABSTRACT

Maurer and Bretz developed a class of group-sequential weighted Bonferroni procedures with multiple endpoints. Performed as a step-down consonant shortcut to the group-sequential closed testing procedure, the class of procedures of Maurer and Bretz is simple to use for testing multiple endpoints in classical group-sequential settings. This class uses the correlations of sequential statistics, but does not leverage the correlations between endpoints. Thus, there is room for power improvement by suitably using the between-endpoint correlations in a group-sequential trial while maintaining strong control of the family-wise error rate. To this end, we propose a Holm-type step-down exact parametric procedure for situations in which between-endpoint correlations are known a priori or estimable. An adaptive strategy is suggested for situations in which such correlations are unknown. In addition, we briefly discuss a natural group-sequential extension of the partially parametric Seneta–Chen procedure.

KEYWORDS:

• Closure method
• Consonant procedure
• Family-wise error rate
• Group-sequential
• Multiple testing procedure

1. Introduction

The classical group-sequential clinical trial methodologies for testing single hypothesis are well-developed and frequently used in late phase confirmatory trials. In a group-sequential trial, one or more interim analyses and one final analysis are preplanned, and hypothesis testing at each analysis is based on accumulating data collected up to the analysis. Compared with a fixed-sample trial, a group-sequential trial, when properly implemented, may provide savings in time and sample size because the trial can be stopped earlier for efficacy upon the rejection of the hypothesis at an interim analysis. Repeated significance testing at a fixed level on accumulating data during the course of a trial will greatly inflate the Type I error rate. To avoid this problem, a valid group-sequential testing method uses a set of appropriately determined critical boundaries such that the overall Type I error rate is controlled at the nominal level. There is a large body of literature on the topic. Jennison and Turnbull (2000) provided a comprehensive exposition of the methods.

Recent years have seen the development of a class of group-sequential closed test procedures based on the weighted Bonferroni test (Maurer and Bretz 2013) for testing multiple endpoints (hypotheses) in clinical trials, where the objective is to reject at least one endpoint (hypothesis) while maintaining strong control of the family-wise error rate at a prespecified level α across all analysis time points and endpoints. Such a class of procedures has the following desirable properties: (1) it conveniently uses marginal test statistics, which are the individual test statistics corresponding to the individual endpoints; (2) it is sequentially rejective at each of the analysis time points; and (3) it provides a generalization in that the popular group-sequential fixed sequence procedure (e.g., Strategy 2 in Hung, Wang, and O’Neill 2007; Strategy 1 and Strategy 2 in Tamhane, Mehta, and Liu 2010) and the recently proposed group-sequential Holm procedure (the method GSHv in Ye et al. 2013) are special members of the class. Note that the terms “endpoint” and “hypothesis” are used interchangeably in this article.

Although a powerful α-propagation (also called “recycling” or “transferring”) rule is incorporated, the class of procedures (Maurer and Bretz 2013) does not consider between-endpoint correlations, which we refer to as correlations of the test statistics associated with the individual clinical endpoints. In a group-sequential trial with correlated multiple endpoints, power improvement can be achieved by suitably using the between-endpoint correlations. In situations where such correlations are nonnegligibly high, the proposed parametric procedure will provide substantial power improvement that can translate to a reduced sample size for a given power requirement. Many parametric procedures are available, and they partly depend on the specific test statistics which one can choose. For example, the parametric group-sequential procedure of Tang and Geller (1999) uses the centered linear combination statistics. Our interest here is to propose parametric procedures that retain the sequential rejectiveness property of the original Holm procedure (1979) in the group-sequential settings. This desirable property is referred to as consonancy in the theoretical framework of the closed testing principle (Marcus, Peritz, and Gabriel 1976). The group-sequential parametric procedure of Tang and Geller (1999) is exact, but not consonant.

The goal we set up in this article for the multiple testing procedures is to reject as many endpoints as possible while strongly preserving the family-wise error rate at a prespecified level. Thus, the testing is assumed to be continued until the rejection of all endpoints or the completion of the trial, whichever occurs earlier. We start by presenting a perspective on the classical group-sequential methodology for single hypothesis, and introducing some notations.

2. Univariate Critical Boundary for Single-Hypothesis

The error spending approach (Lan and DeMets 1983) is a convenient tool for group-sequential design and analysis. Several specific forms of error spending functions have been proposed, and a few are commonly used in clinical applications. To accommodate the problem of testing multiple hypotheses, Maurer and Bretz (2013) attempted some slight generalizations of spending functions, and they redefined a family of error spending functions $\mathcal{F}(\alpha ,r)$ as follows: Parameterized by a prespecified Type I error rate α, 0 < α < 1, $\mathcal{F}(\alpha ,r)$ is a cumulative Type I error spending function of the information fraction r, 0 ⩽ r ⩽ 1. They further suggested that mild monotonicity conditions should be imposed: (1) for a given α, $\mathcal{F}(\alpha ,r)$ is nondecreasing in r, with $\mathcal{F}(\alpha ,0)=0$ and $\mathcal{F}(\alpha ,1)=\alpha$; and (2) for a given r, $\mathcal{F}(\alpha ,r)$ is nondecreasing in α. A family of spending functions that are nonmonotonic may not be admissible for use. Commonly used error spending functions, such as the γ-family (Hwang, Shih, and DeCani 1990), the power family of error spending functions (Jennison and Turnbull 2000, p. 148), and the approximated O’Brien-Fleming and Pocock error spending functions (Lan and DeMets 1983) are strictly monotone in both r and α,  α ∈ (0, α⁰), where  0 < α⁰ < 1. Note that practically α⁰ can be far less than 1, but usually needs to exceed 0.05 or 0.10, which are the commonly used significance levels in hypothesis testing.

Consider the problem of testing single hypothesis H_i based on a sequence of standardized test statistics (Z_{i, 1}, …, Z_{i, T}) in a T-stage group-sequential setting. Let δ_i be the parameter for treatment effect. A commonly used distribution model is that (Z_{i, 1}, …, Z_{i, T}) is a multivariate standard normal distribution with sequential means $E\left(Z_{i,t}\right)={\delta }_i\sqrt{n_t}$ and sequential correlations $\text{corr}(Z_{i,t},Z_{i,t^{\text{'}}})=\sqrt{n_t/n_{t^{\text{'}}}},1\le t<t^{\text{'}}\le T$, where n_t is the number of subject pairs (half of the sample size) at stage t for a comparative trial with equal allocation to two treatment arms. Let P_{i, t} be the p-value corresponding to the test statistics Z_{i, t} based on the cumulative data collected up to the time of analysis t, t = 1, …, T. The group-sequential test rejects H_i if, for any t = 1, …, T, P_{i, t} ⩽ b_{i, t}, where b_{i, t} is the stage t univariate critical boundary expressed in terms of significance level. In the following, we review the derivation of the classical univariate boundary {b_{i, t}, t = 1, …, T}.

Let ${\mathcal{F}}^i$ be a continuous and strictly monotone error spending function that is prescribed to H_i. The allocation of spent levels {α₁, …, α_t, …, α_T} across T analyses is immediately obtainable, with α_t ≠ 0 for analysis t, t = 1, …, T, and ∑^T_{t = 1}α_t = α. Then, solve the following set of equations for the univariate (single hypothesis) critical boundary {b_{i, t},  t = 1, …, T} under the null hypothesis. (1) $$\begin{array}{ccc}& & \mathrm{pr}(P_{i,1}<b_{i,1})={\alpha }_1\hfill \\ & & \mathrm{pr}(\{P_{i,1}\ge b_{i,1}\}\cap \{P_{i,2}<b_{i,2}\})={\alpha }_2\hfill \\ & & ⋮\hfill \\ & & \mathrm{pr}({\cap }_{k=1}^{t-1}\{P_{i,k}\ge b_{i,k}\}\cap \{P_{i,t}<b_{i,t}\})={\alpha }_t\hfill \\ & & ⋮\hfill \\ & & \mathrm{pr}({\cap }_{k=1}^{T-1}\{P_{i,k}\ge b_{i,k}\}\cap \{P_{i,T}<b_{i,T}\})={\alpha }_T.\hfill \\ & & \hfill \end{array}$$(1)

The solution {b_{i, t}, t = 1, …, T} to (1) can be treated in principle as an implicit vector function of α,  α ∈ (0, α⁰), where 0 < α⁰ < 1. Thus, we replace b_{i, t}(α) by a more suggestive notation ${\mathcal{F}}_{i,t}\left(\alpha \right)$. The vector function $\{{\mathcal{F}}_{i,1}\left(\alpha \right),\cdots ,{\mathcal{F}}_{i,T}\left(\alpha \right)\}$ denotes the ${\mathcal{F}}^i$-induced univariate level-α group-sequential critical boundary for testing H_i. Further, it can be shown that $\{{\mathcal{F}}_{i,1}\left(\alpha \right),\cdots ,{\mathcal{F}}_{i,T}\left(\alpha \right)\}$ is monotonically increasing in α (Maurer and Bretz 2013).

Let S_{i, t}(α) denote the event $\{P_{i,t}<{\mathcal{F}}_{i,t}\left(\alpha \right)\}$. Then, the critical region of the classical group-sequential test of H_i is ∪^T_{t = 1}S_{i, t}(α). Let R_i(α) = {P_i ⩽ α}, where P_i is distributed as U(0, 1). Then the equivalency of two events is (2) $$R_i\left(\alpha \right)\equiv {\cup }_{t=1}^T{S}_{i,t}\left(\alpha \right).$$(2)

That is, the critical region of a group-sequential test can be seen as equivalent to that of a hypothetically constructed fixed-sample reference test. It follows that equivalent events have equal probabilities.

3. Parametric Test for Intersection Hypothesis

3.1. Fixed-Sample Parametric Test

Consider a fixed-sample parametric test for an intersection hypothesis before considering the group-sequential extension by analogy.

For the fixed-sample testing, the parametric procedure of Xie (2012) implicitly uses a Holm-type weighted parametric test. The Xie procedure can also be viewed as a weighted version of the original Dunnett and Tamhane (1991) step-down procedure using Holm’s weighting scheme. Xie’s (2012) parametric test is based on the use of adjusted p-values whereas ours, as demonstrated in the following, is based on adjusted significance levels. We choose to present our formulation of the parametric test, which is more conducive to its group-sequential extension.

Holm’s weighting scheme is illustrated in Holm’s (1979) paper. Let I = {1, …, m} denote the index set of m null hypotheses. For I, assume a collection of initial weights w_i(I), i ∈ I, with 0 < w_i(I) ⩽ 1 and ∑_{i ∈ I}w_i(I) = 1. The procedure starts by comparing each p_i with w_i(I)α at Step 1, and it rejects the hypotheses in the set R = {H_i, i ∈ I: p_i < w_i(I)α}. If R = ∅, then the test procedure stops without a rejection. Otherwise, the test proceeds to Step 2, and nonrejected hypotheses in the set I∖R are retested using updated larger weights w_i(I∖R) = w_i(I)/(1 − ∑_{r ∈ R}w_r(I)), i ∈ I∖R. In Step 2, the procedure compares each p_i with the larger critical value w_i(I∖R)α to make a decision about rejection. Testing continues with updated weights for the non-rejected hypotheses until the first nonrejection.

To test an intersection hypothesis at a prespecified significance level α, a parametric test uses the joint null distribution of test statistics, which results in a larger critical region than that of the weighted Bonferroni test. Consider the testing of an intersection hypothesis $H_J={\cap }_{i\in J}{H}_i$, where J⊆I = {1, 2, …, m}. The parametric test rejects H_J if, for some i, i ∈ J, p_i < ξ_Jw_i(J)α, where w_i(J) > 0 is the weight for H_i and ξ_J ⩾ 1 is the solution to the following equation (Bretz et al. 2011). Note that the probability is evaluated under the global (intersection) null hypothesis: (3) $$\mathrm{pr}\left({\cap }_{i\in J}\{P_i\ge {\xi }_J{w}_i\left(J\right)\alpha \}\right)=1-\alpha .$$(3)

Lemma 3.1.

Holm’s weighting scheme when employed by α-exhaustive parametric tests for all subset intersection hypotheses ensures that the monotonicity condition (Bretz et al. 2011, p. 901) defined as ${\xi }_J{w}_i\left(J\right)\le {\xi }_{J^{\text{'}}}{w}_i\left(J^{\text{'}}\right)$ for all J′⊆J⊆I and i ∈ J′ is satisfied.

The proof is given in the appendix.

Remark 3.1.

Lemma 1 can be stated for commonly used model settings, such as multivariate normal, absolute multivariate normal, multivariate t, and absolute multivariate t-test statistic; however, a particular distributional assumption is not required. Moreover, equal correlations that imply the exchangeability of the null p-values are not required either.

3.2. Group-Sequential Parametric Test

In this section, we construct a group-sequential parametric test for an intersection null hypothesis $H_I={\cap }_{i\in I}{H}_i$, where I = {1, 2, …, m}.

Assume a collection of weights {w_i(I), i ∈ I} and a collection of error spending functions $\{{\mathcal{F}}^i,i\in I\}$, with w_i and ${\mathcal{F}}^i$ prescribed for H_i. Also assume for each H_i, i ∈ I, a sequence of standardized test statistics (Z_{i, 1}, …, Z_{i, T}), which is multivariate standard normal with the sequential means and the sequential correlation structure described in Section 2. Further assume for each t, Z_{i, t} and $Z_{i^{\text{'}},t}$ are jointly bivariate normal with a known correlation ${\rho }_{i,i^{\text{'}}}$ between H_i and $H_{i^{\text{'}}}$. Note that correlation ${\rho }_{i,i^{\text{'}}}$ between H_i and $H_{i^{\text{'}}}$ is in fact the same for all t, t = 1, …, T, and it does not vary from t to t′ under the multivariate normal model. The correlation structure for the m endpoints, which is usually expressed by an m × m matrix, will be used in the boundary calculations for our proposed parametric procedures. We should mention that correlations exist for test statistics between different endpoints and different stages. Fortunately, as shown later, the proposed α-exhaustive (exact) parametric procedure does not require knowing such correlations for boundary calculations. We consider this feature an advantage with respect to simplicity.

A T-stage group-sequential parametric test is simply constructed as follows. This test rejects H_I if there exist an i ∈ I and a t, t = 1, …, T such that $P_{i,t}\le {\mathcal{F}}_{i,t}\left({\xi }_I{w}_i\left(I\right)\alpha \right)$, where ξ_I is the solution to Equation (3). Otherwise, the test does not reject H_I.

Lemma 3.2.

The above described T-stage group-sequential parametric test for an intersection null hypothesis $H_I={\cap }_{i\in I}{H}_i$ is a size-α test.

Proof 3.1.

The lemma can be verified by calculating the probability of rejecting H_I under the global (intersection) null hypothesis. $$\begin{array}{ccc}& & \mathrm{pr}\left({\cup }_{t=1}^T{\cup }_{i=1}^m\{P_{i,t}\le {\mathcal{F}}_{i,t}\left({\xi }_I{w}_i\left(I\right)\alpha \right)\}\right)(\mathrm{where}{\xi }_{\mathrm{I}}\mathrm{is}\mathrm{the}\mathrm{exact}\hfill \\ & & \mathrm{solution}\mathrm{to}\mathrm{Equation}\left(3\right))\hfill \\ & & =\mathrm{pr}\left({\cup }_{i=1}^m{\cup }_{t=1}^T\{P_{i,t}\le {\mathcal{F}}_{i,t}\left({\xi }_I{w}_i\left(I\right)\alpha \right)\}\right)\hfill \\ & & =\mathrm{pr}\left({\cup }_{i=1}^m{\cup }_{t=1}^T{S}_{i,t}\left({\xi }_I{w}_i\left(I\right)\alpha \right)\right)\hfill \\ & & =\mathrm{pr}\left({\cup }_{i=1}^m{R}_i\left({\xi }_I{w}_i\left(I\right)\alpha \right)\right)(\mathrm{obtained}\mathrm{by}\mathrm{applying}\mathrm{the}\mathrm{result}\hfill \\ & & \mathrm{in}\left(2\right),\mathrm{which}\mathrm{concerns}\mathrm{equivalency})\hfill \\ & & =\alpha \left(\mathrm{which}\mathrm{is}\mathrm{essentially}\mathrm{implied}\mathrm{by}\mathrm{Equation}\right(3\left)\right).\hfill \end{array}$$

Note that Lemma 3.2 also applies to situations in which different families (types) of error spending functions are prespecified for different hypotheses.

We next show that, under Holm’s weighting scheme, the monotonicity property exhibited in the fixed-sample parametric closed testing procedure is retained in its group-sequential extension.

Lemma 3.3.

The described T-stage group-sequential parametric tests for the family of all subset intersection hypotheses $\{H_J={\cap }_{i\in J}{H}_i,J\subseteq I\}$ have the desirable property of monotonicity of critical values at each analysis time point t = 1, …, T.

Proof 3.2.

Without a loss of generality, let i ∈ J′⊂J⊆I. That is, let H_i be a component hypothesis in both $H_{J^{\text{'}}}$ and H_J. $$\begin{array}{ccc}& & w_i\left(J\right)<w_i\left(J^{\text{'}}\right)\text{(per}\text{Holm"s}\text{weighting}\text{scheme)}\hfill \\ & & ⟹{\xi }_J{w}_i\left(J\right)<{\xi }_{J^{\text{'}}}{w}_i\left(J^{\text{'}}\right)\text{(per}\text{Lemma}\text{1)}\hfill \\ & & ⟹{\mathcal{F}}_{i,t}\left({\xi }_J{w}_i\left(J\right)\alpha \right)<{\mathcal{F}}_{i,t}\left({\xi }_{J^{\text{'}}}{w}_i\left(J^{\text{'}}\right)\alpha \right),\text{for}\text{each}\text{analysis}\text{time}\hfill \\ & & \mathrm{point}t=1,\cdots ,T\hfill \\ & & \text{(per}\text{the}\text{employment}\text{of}\text{a}\text{strictly}\text{monotone}\text{spending}\hfill \\ & & \mathrm{function}{\mathcal{F}}^{\mathrm{i}}\text{for}{\mathrm{H}}_{\mathrm{i}}).\hfill \end{array}$$

4. Group-Sequential Parametric Procedure for Multiple Endpoints

In this section, we assume that the correlation matrix for the m endpoints is known or estimable. To utilize these between-endpoint correlations in perhaps the simplest way possible, we construct a group-sequential parametric procedure that has the same feature as the weighted Bonferroni procedures (Maurer and Bretz 2013) in that it is step-down, sequentially rejective at each analysis time point, requiring at most max (m, T) tests to finish the testing of m hypotheses. In the following, we outline the algorithm for performing the proposed T-stage group-sequential parametric procedure with m endpoints.

Algorithm

0. Set t = 1, I = {1, 2, …, m}, {w_i(I), i ∈ I}, and $\{{\mathcal{F}}^i,i\in I\}$.

1. Calculate ξ_I from Equation (3), and then calculate the critical boundaries (nominal significance levels) $\{{\mathcal{F}}_{i,t}\left({\xi }_I{w}_i\left(I\right)\alpha \right),i\in I\}$.

2. Construct an index set $R=\{i\in I:p_{i,t}<{\mathcal{F}}_{i,t}\left({\xi }_I{w}_i\left(I\right)\alpha \right)\}$, where p_{i, t} is the unadjusted p-value for H_i at analysis t.

3. If R ≠ ∅, then reject H_i, i ∈ R. Update I → I∖R and w_i(I) → w_i(I∖R). Go to Step 1.

4. If R = ∅, and t < T, then the trial is continued with t → t + 1. Go to Step 2; otherwise stop.

5. If |I| ⩾ 1, go to Step 1; otherwise stop.

Theorem 4.1.

The proposed group-sequential weighted parametric multiple testing procedure strongly controls the family-wise error rate.

Proof 4.1.

Immediate from applying the closure method to the group-sequential α-exhaustive parametric tests for all intersection hypotheses $H_J={\cap }_{i\in J}{H}_i,J\subseteq I$. Note that the intersection tests are α-exhaustive parametric tests (from Lemma 2), and the intersection tests have a monotone property (from Lemma 3).

Ye et al. (2013) proposed two variants of their group-sequential Holm’s method, termed “group-sequential Holm variant (GSHv)” and “group-sequential Holm fixed (GSHf).” The distinction between the two variants is as follows. Suppose that the first rejection of an endpoint is made at t, t ⩽ T for a T-stage group-sequential trial. If the group-sequential critical boundaries (for those unrejected endpoints) are instantly updated for all stages 1, …, T, but effectively changed only for stages t, …, T, the procedure is termed GSHv; if the boundaries are updated only for the final stage T, the procedure is termed GSHf. The variant GSHv is obtainable simply by setting ξ_J = 1 for all J⊆I in Equation (3) and, thus, represents a special case of our proposed group-sequential parametric procedure, because ξ_J tends to 1 as all ρs tend to − 1/(m − 1). It can be shown that ξ_J cannot be less than 1.

Recently, Xi and Tamhane (2015) generalized the time at which boundary updates should be made upon the first rejection at t, and they proposed that the updating can be performed for stages s*, …, T, where 1 ⩽ s* ⩽ T. In their generalization, if s* = 1, then the procedure is the same as GSHv; if s* = T, then the procedure is the same as GSHf. Xi and Tamhane (2015) further suggested methods for choosing s* to minimize the expected sample size for the given power requirement.

5. Simulation

We conducted simulation studies to investigate how our proposed parametric procedure compares with the Holm procedure in terms of the family-wise error rate and the average power, which is the expected proportion of rejected false null hypotheses, for one-sided tests using multivariate normal test statistics. Because the variant GSHv of the Holm procedure (Ye et al. 2013) is a special case of the proposed parametric procedure, we use it as a benchmark. For simplicity, we based the simulation on a two-stage (T = 2) group-sequential setting with the following assumptions: (1) equicorrelated multivariate normal test statistics with a mean μ_i for H_i and a common correlation coefficient ρ among endpoints; (2) equispaced information fractions (i.e., equal sample size for stages 1 and 2); and (3) same types of spending functions for all the endpoints. Note that both of the competing procedures are applicable if different types of error spending functions are prescribed for different endpoints, as long as the type that is prespecified for each endpoint remains unchanged during the course of the analysis.

All computations and simulations were conducted using R version 3.3.2. Without exception, all boundaries were conveniently calculated in terms of the nominal significance levels by invoking the R function gsDesign (Anderson 2016). This function returns boundary values to four decimal places.

We considered different values of m from 2 to 8, and did not attempt a larger value of m, because individual boundaries returned by the R function for smaller spending levels resulting from a larger m lack the accuracy required for group-sequential simulations.

Figure 1, which shows the relative performances of four methods for m = 5 equally weighted endpoints, is a representative of how our methods perform for m ⩾ 2. The four methods are as follows: the regular Holm procedure based on the O’Brien-Fleming error spending function (Holm-OBF), the parametric Holm procedure based on the O’Brien-Fleming function (Para-OBF), the regular Holm procedure based on the Pocock function (Holm-PO), and the parametric Holm procedure based on the Pocock function (Para-PO). Figure 1 shows that, for a given error spending function, the proposed parametric group-sequential method uniformly outperforms the regular Holm’s group-sequential method, especially when the between-endpoint correlation ρ is high. Similar results are seen in Figure 2 for a slightly different simulation configuration in which different hypotheses take different means.

Figure 1. Comparison of four methods, Holm-OBF (red dotted), Para-OBF (red solid), Holm-PO (blue dotted), and Para-PO (blue solid), for testing μ_i ⩽ 0 against μ_i > 0, simultaneously for i = 1, …, m = 5, at level α = 0.05 based on multivariate normal test statistics with the common correlation ρ. The mean is chosen to be 2 when a null hypothesis is false, and 0 when a null hypothesis is true, with π being the proportion of true null hypotheses. One million independent replications were used in all simulations.

Figure 2. Comparison of four methods, Holm-OBF (red dotted), Para-OBF (red solid), Holm-PO (blue dotted), and Para-PO (blue solid), for testing μ_i ⩽ 0 against μ_i > 0, simultaneously for i = 1, …, m = 5, at level α = 0.05 based on multivariate normal test statistics with the common correlation ρ. The means are chosen to be 1.0, 1.5, and 2.0, respectively, for the 3 false null hypotheses, and − 1.0 and 0 for the 2 true null hypotheses, with π being the proportion of true null hypotheses. One million independent replications were used in all simulations.

6. Application and Example

In the real-world clinical trial practice, estimating the correlations among endpoints is generally difficult . However, situations occur in which such correlations are structural. Multiple composite endpoints are widely used in large-scale cardiovascular Phase III confirmatory trials, which typically enroll tens of thousands of subjects. For example, the MERIT-HF trial (Fagerberg 1999) is a group-sequential trial with two primary endpoints, which are (1) all-cause mortality and (2) the composite of all-cause mortality plus all-cause hospitalization. The trial also has four secondary composite endpoints, which include common components. We speculate that with the addition of certain reasonable assumptions, correlations of the log-rank test statistics for composite endpoints with common components can be appropriately estimated. Further research will be needed with respect to such estimations.

The proposed group-sequential parametric step-down procedure is also applicable to the problem of testing multiple hypotheses of the same endpoint, but associated with different populations of interest. These different populations overlap to some degree.

6.1. Example with Calculations Using R

We revisited the group-sequential MONET1 study, which enrolled a total of n = 1060 subjects (Section 4, Ye et al. 2013). This study is a classical two-stage (one interim and one final) group-sequential trial that uses equispaced information fractions. The study has two primary hypotheses H₁ and H₂, both of which concern the treatment effect on overall survival (represented by hazard ratio λ). H₁ concerns this treatment effect in the adenocarcinoma subpopulation (70% of the total population), and H₂ concerns this treatment effect in the total population. The allocated α is 0.010 for testing H₁ and 0.015 for testing H₂, both one-sided. For illustration purposes, Ye et al. (2013) used the O’Brien-Fleming error spending function for both H₁ and H₂. The log rank test statistics are used. Under the null hypothesis log(λ) = 0, the variance of log rank statistics is approximated as 4/q_t (Jennison and Turnbull 2000, pp. 77–79), where q_t is the number of cumulative deaths at analysis t. This approximation holds under equal randomization, which is the case of the MONET1 study with the randomization ratio 1:1 (Chemotherapy:Placebo). The correlation of the two test statistics is high, which is a result of the large portion of the population that overlaps. Under the additional assumption that death rates are comparable among the total population and the adenocarcinoma subpopulation, the correlation coefficient of the Wald test statistics associated with H₁ and H₂ is estimated to be $\sqrt{0.70}=0.837$.

The critical values (in terms of z-score) of the group-sequential Holm procedure are tabulated (see the GSHv row of Table 1, Ye et al. 2013) as follows: c₁ = 3.25, c′₁ = 2.96, c₂ = 2.18, and c′₂ = 1.97 in the overall population (H₂), and d₁ = 3.46, d′₁ = 2.96, d₂ = 2.33, and d′₂ = 1.97 in the adenocarcinoma subpopulation (H₁), where c_t and c′_t, t = 1, 2, are two-stage group-sequential boundaries for H₂ at the significance levels 0.015 and 0.025 respectively; and d_t and d′_t, t = 1, 2, for H₁ at the significance levels 0.010 and 0.025, respectively. These notations are the same as those used by Ye et al. (2013). We successfully replicated the above critical values, which are based on the approximated O’Brien-Fleming error spending function of Lan and DeMets (1983).

The proposed parametric procedure calculates the boundaries as follows. By using the estimated correlation 0.837, we obtain ξ = 1.28, which is then used to calculate the following exact boundaries: c₁ = 2.95, c′₁ = 2.65, c₂ = 2.08, and c′₂ = 1.87 in the overall population; and d₁ = 3.17, d′₁ = 2.65, d₂ = 2.24, and d′₂ = 1.87 in the adenocarcinoma subpopulation. The proposed parametric procedure has a uniformly larger critical region, which is apparent from the above numbers. Moreover, the parametric procedure’s critical region is considerably larger since ξ = 1.28 > 1. Note that 1 is the value that ξ takes for the GSHv variant of the group-sequential Holm procedure. We provide the R code in the appendix.

6.2. Adaptive Strategy for Unknown Correlations

The proposed group-sequential parametric method is preferable when the correlation structure for the multiple endpoints is known a priori because solving for the constant ξ_J to Equation (3) is made easy by modern computing. For a given α, the higher the between-endpoint correlations ρ's, the higher the calculated value of ξ_J, and the larger the group-sequential critical regions for the unrejected endpoints. In practice, a complete set of correlations among endpoints is often impractical to determine, or simply the confidence in the presumably known correlations may be weak. In such situations, correlations can be estimated for each stage t based on the data collected up to stage t. The estimated correlations are then used to calculate the values of ξ_J, and then boundaries, by which decisions are made about what to reject. This adaptive parametric procedure maintains strong control of the family-wise error rate asymptotically because sample estimates of the correlations tend to approach the true correlations asymptotically. For the same reason, this adaptive parametric method does not work well for smaller samples. The classical group-sequential test is based on the cumulative data, and correlation estimates become more reliable for the later stages of the trial. Further work is needed to evaluate the performance of the adaptive parametric procedure in the group-sequential applications.

Tang and Geller (1999) provided an example of a cross-over trial with four respiratory endpoints of interest (FEV1, FVC, PEFR, and PI). The first part of the cross-over trial is treated as if it is the first stage of a two-stage group-sequential trial. The following is the estimated correlation matrix (Tang and Geller 1999, p. 1189) for stage 1: $$\begin{array}{c}\hfill \left[\begin{array}{cccc}\hfill 1& \hfill 0.095& \hfill 0.219& -0.162\\ & \hfill 1& \hfill 0.518& -0.059\\ & & \hfill 1& 0.513\\ & & & 1\end{array}\right].\end{array}$$

Now suppose that the proposed consonant parametric procedure is used to perform simultaneous one-sided testing of the mean differences for the four equally weighted endpoints at the overall level α = 0.05. By solving Equation (3), we obtain ξ_I = 1.11 for I = {1, 2, 3, 4}. The initial boundaries are 0.0009 (O’Brien-Fleming) and 0.0079 (Pocock) using the ξ_I value, and 0.0007 (O’Brien-Fleming) and 0.0072 (Pocock) if the variant GSHv of the group-sequential Holm procedure (Ye et al. 2013) is used. Note that each of the four individual p-values for the first stage is to be compared with the initial boundaries to make the initial decision about rejection. This example shows that although the correlations among endpoints may be small on average, or even negative as seen in the above matrix, the impact on the critical boundaries by taking them into account can be nonnegligible.

7. Discussion

The class of group-sequential weighted Bonferroni procedures (2013) is a consonant step-down case of the generic group-sequential closed testing procedures (Tang and Geller 1999). The consonancy generates a shortcut to the fully closed procedure, and does not require greater than max (m, T) tests to finish the testing, thus, representing a considerable simplification of the generic group-sequential closed testing procedure (Tang and Geller 1999) that may require as many as T2^m tests.

A Bonferroni-based closed testing procedure requires little restrictions on the allocation and reallocation of weights for the procedure to be consonant. Thus, there is an unconstrained freedom in selecting a particular weighting scheme to suit a specific study objective. However, to create consonancy in the parametric-based closed testing procedure, one has to resort to certain specific weighting schemes, such as Holm’s weighting scheme. Holm’s weighting scheme preserves the initial ratios of nonzero weights, which may represent the relative importance of the individual hypotheses. In clinical trials with multiple primary endpoints, Holm’s weighting scheme has a prominent place in suiting certain specific study objectives. A commonly used strategy is to base the initial weights assigned to individual endpoints on their relative importance. A different strategy is to choose the initial weights to maximize the disjunctive power, given the expected effect sizes for all endpoints.

Because of its particular underlying weighting scheme, the fallback-type parametric procedure (Huque and Alosh 2008) for hierarchically ordered endpoints cannot be easily extended for applications in group-sequential settings. For m ⩾ 3, the monotonicity condition (Bretz et al. 2011) does not hold for positive correlations, although it holds for the cases of independent or negative correlations.

In situations where the correlations between endpoints are known for some but not for all, Bretz et al. (2011) provided a solution, which requires using a submatrix of those fully known correlations for solving Equation (3). It can be shown that their solution preserves consonancy in a consonant parametric procedure. However, the disadvantage is apparent, because not all known pairwise correlations ${\rho }_{i{i}^{\text{'}}}$ between H_i and $H_{i^{\text{'}}}$, i, i′ ∈ I are used. For an example, consider the testing of three correlated endpoints H₁, H₂, and H₃, with ρ₁₂ and ρ₂₃ known while ρ₁₃ unknown. The solution of Bretz et al. (2011) uses either ρ₁₂ or ρ₂₃, presumably the larger one, but not both. An alternative solution is to use the partially parametric procedure of Seneta and Chen (2005), which uses all known pairwise correlations in boundary calculations, thus, can be more powerful. Seneta and Chen procedure (2005) is a (consonant) step-down procedure. The procedure is not exact, but has the desirable property of allowing explicit formulas for the adjusted critical boundaries. Note that the fixed-sample Seneta and Chen (2005) procedure is originally unweighted (or equally weighted), but its weighted version is readily available by applying Holm’s weighting scheme. A natural group-sequential extension of the weighted Seneta and Chen method can be achieved by employing monotone error spending functions, because the monotonicity condition referred to in Lemma 1 is preserved for each of the analysis time points. Sarkar, Fu, and Guo (2016) improved the method of Seneta and Chen (2005) using a convexity property of pairwise null p-values without increasing the computational complexity. These Holm-type step-down partially parametric methods can be extended to the group-sequential setting, and their group-sequential extensions strongly control the family-wise error rate and are generally more powerful than alternatives that do not use the correlations among endpoints.

If the correlations between endpoints are independent or positive, which is commonly observed in clinical trials, there exists an applicable group-sequential step-up procedure that is strictly more powerful than the group-sequential Holm procedure, and also maintains strong control of the family-wise error rate, and does not exceed the complexity of the Holm procedure. The group-sequential step-up procedure for testing correlated endpoints will be elaborated in a future communication.

Supplementary Material

Supplementary material available includes simulation R codes and outputs.

Acknowledgments

The author thanks Dr. Frank Bretz and Professor Ajit Tamhane for their valuable comments. The author acknowledges the constructive comments of editor Dr. José Pinheiro, the associate editor, and four referees, who tremendously helped to enhance the ideas and improve the accuracy and the clarity of this manuscript.

Appendix

Proof of Lemma 1 in Section 3.1

Proof.

Without loss of generality, assume that J′ is a proper subset of J; that is, J′ = J∖R, where R ≠ ∅. Let ${\xi }_{J^{\text{'}}}$ be the solution to (A.1) $$\begin{array}{c}\hfill \mathrm{pr}\left({\cap }_{i\in J\setminus R}\{P_i\ge {\xi }_{J^{\text{'}}}{w}_i(J\setminus R)\alpha \}\right)=1-\alpha ,\end{array}$$(A.1) where w_i(J∖R) = w_i(J)/(1 − ∑_{r ∈ R}w_r(J)), for i ∈ J∖R.

From (3) and (4), we have (A.2) $$\begin{array}{c}\hfill \mathrm{pr}\left({\cap }_{i\in J}\{P_i\ge {\xi }_J{w}_i\left(J\right)\alpha \}\right)=1-\alpha =\mathrm{pr}\left({\cap }_{i\in J\setminus R}\{P_i\ge {\xi }_{J^{\text{'}}}{w}_i(J\setminus R)\alpha \}\right),\end{array}$$(A.2) and by plugging $\mathrm{pr}\left({\cap }_{i\in J\setminus R}\{P_i\ge {\xi }_J{w}_i\left(J\right)\alpha \}\right)\ge \mathrm{pr}\left({\cap }_{i\in J}\{P_i\ge {\xi }_J{w}_i\left(J\right)\alpha \}\right)$ into (5), we have (A.3) $$\begin{array}{ccc}& & \mathrm{pr}\left({\cap }_{i\in J\setminus R}\{P_i\ge {\xi }_J{w}_i\left(J\right)\alpha \}\right)\ge \mathrm{pr}\left({\cap }_{i\in J\setminus R}\{P_i\ge {\xi }_{J^{\text{'}}}{w}_i(J\setminus R)\alpha \}\right).\hfill \end{array}$$(A.3) (A.4) $$\begin{array}{ccc}& & \text{Let}\eta =\frac{{\xi }_{J^{\text{'}}}{w}_i(J\setminus R)\alpha }{{\xi }_J{w}_i\left(J\right)\alpha }=\frac{{\xi }_{J^{\text{'}}}}{{\xi }_J(1-{\sum }_{r\in R}{w}_r\left(J\right))},\text{which}\text{is}\text{a}\text{constant}\hfill \\ & & foralli\in J\setminus R.\hfill \end{array}$$(A.4) Replacing the right side of the inequality (6) by the η expression from (7), we get (A.5) $$\begin{array}{c}\hfill \mathrm{pr}\left({\cap }_{i\in J\setminus R}\{P_i\ge {\xi }_J{w}_i\left(J\right)\alpha \}\right)\ge \mathrm{pr}\left({\cap }_{i\in J\setminus R}\{P_i\ge \eta ·{\xi }_J{w}_i\left(J\right)\alpha \}\right).\end{array}$$(A.5) If  η < 1, we have {P_i ⩾ ξ_Jw_i(J)α}⊂{P_i ⩾ η · ξ_Jw_i(J)α}, for i ∈ J∖R, leading to $\mathrm{pr}\left({\cap }_{i\in J\setminus R}\{P_i\ge {\xi }_J{w}_i\left(J\right)\alpha \}\right)<\mathrm{pr}\left({\cap }_{i\in J\setminus R}\{P_i\ge \eta ·{\xi }_J{w}_i\left(J\right)\alpha \}\right)$, which is a contradiction to the inequality (8). It then follows that η ⩾ 1 for all i ∈ J′.

R code for Section 6 $$\begin{array}{ccc}& & library(``mvtnor{m}^{\text{'}\text{'}})\hfill \\ & & dunn<-function(c,w1,w2,alpha,rho)\{\hfill \\ & & mean<-rep(0,2)\hfill \\ & & corr<-diag\left(2\right)\hfill \\ & & corr[lower.tri(corr\left)\right]<-rho\hfill \\ & & corr[upper.tri(corr\left)\right]<-rho\hfill \\ & & z<-pmvnorm(lower=-Inf,\hfill \\ & & upper=c\left(qnorm\right(1-alpha*w1*c),qnorm(1-alpha*w2*c\left)\right),\hfill \\ & & mean=mean,\hfill \\ & & corr=corr)\hfill \\ & & nonlin<-1-z-alpha\hfill \\ & & return\left(nonlin\right)\}\hfill \\ & & \\ & & y2<-function(w1,w2,alpha,rho)\{\hfill \\ & & z<-uniroot(dunn,lower=1,upper=9,w1=w1,w2=w2,alpha\hfill \\ & & =alpha,rho=rho\left)\$rootreturn\right(z\left)\right\}\hfill \\ & & \\ & & round\left(y2\right(w1=0.40,w2=0.60,alpha=0.025,rho=0.837),\hfill \\ & & digits=2)1.28\hfill \\ & & library(``gsDesig{n}^{\text{'}\text{'}})\hfill \\ & & w1<-0.4\hfill \\ & & w2<-0.6\hfill \\ & & xsi<-1.28\hfill \\ & & alpha0<-0.025\hfill \\ & & gsDesign\left(k=2,test.type=1,sfu=``OF{,}^{\text{'}\text{'}}alpha=xsi*alpha0\right)\hfill \\ & & gsDesign\left(k=2,test.type=1,sfu=``OF{,}^{\text{'}\text{'}}alpha=w1*xsi*alpha0\right)\hfill \\ & & gsDesign\left(k=2,test.type=1,sfu=``OF{,}^{\text{'}\text{'}}alpha=w2*xsi*alpha0\right)\hfill \end{array}$$

Note that sfu = ``OF′′ in the above coding stands for the original O’Brien-Fl eming error spending method (O'Brien and Fleming 1979).