Embedding a COVID-19 group sequential clinical trial within an ongoing trial: lessons from an unusual experience

Abstract

The Cape-Covid trial is an embedded clinical trial within the ongoing Cape-Cod trial. The Covid-19 pandemic appeared while we were conducting a randomized trial assessing the effectiveness of corticosteroids in severe community-acquired pneumonia. We took advantage of this ongoing trial to embed a sub-trial assessing hydrocortisone in SARS-CoV-2 infected patients. In this manuscript, we wish to share our experience when we needed to make fast and robust methodological decisions during the Covid-19 pandemic in a two weeks period of time.

Keywords:

• SARS-CoV-2 virus
• group-sequential designs

1. Introduction

The COVID-19 pandemic has been disruptive for both standard healthcare and current clinical research. The European Medicines Agency (EMA) recently actualized guidance regarding the management of clinical trials during the COVID-19 pandemic (1), detailing how to handle ongoing trials. The initiation of new trials investigating treatments for COVID-19 received little comment, although researchers are urged to remain rigorous and conduct studies of quality: “crises are no excuse for lowering scientific standards” (2). One situation was however not mentioned, namely the possibility of urgently adjusting an ongoing trial to answer a slightly different question from the one that led to the original study.

The Community-Acquired Pneumonia: Evaluation of COrticosteroiDs (Cape-Cod) trial (NCT02517489) was initiated in 2015. It aims at assessing the efficacy of hydrocortisone on Day-28 all-cause mortality in intensive care unit (ICU) patients suffering from severe community-acquired pneumonia. It is a multicenter two-arm parallel group blinded add-on trial: all patients receive state-of-the-art standard therapy for severe community-acquired pneumonia, including antibiotics and supportive care. The trial aims at recruiting 1,200 patients from 33 centers. Two interim analyses are planned (the first already took place when the pandemics started, with an invitation to continue the trial from the Data Safety Monitoring Board (DSMB), the second will be done for late summer 2020, once data will be collected and checked, which may need a little bit more time than initially planned, due to a potential delay in doing so because of the pandemic). It is funded by the French Ministry of Health. The rational underlying the use of corticosteroids (CTx) in patients suffering from severe community-acquired pneumonia still holds for SARS-CoV-2 infected patients. The need for a randomized clinical trial assessing the efficacy of CTx in severe SARS-CoV-2 infected patients is great (3).

Inclusion criteria into the Cape-Cod trial were valid for SARS-CoV-2 infected patients. Moreover, viral and therefore SARS-CoV-2 pneumonia were not an exclusion criterion for our ongoing trial to the exclusion of influenza pneumonia. The Cape Cod Scientific Committee exchanged views by teleconference on ethical and safety issues. The role of the inflammatory response (cytokine storm) in the respiratory impairment of COVID-19, the efficacy of corticosteroids in non-severe forms of community-acquired pneumonia, including some viral forms, and the uncertainty of their effects in previous coronavirus epidemics, with side effects apparently linked to the high doses used, prompted us to evaluate them in the current epidemic with the rigour of a double-blind randomized trial.

We therefore thought of using the legal, administrative and ethical framework of the active Cape-Cod trial in order to rapidly determine the effectiveness of CTx in severe cases of coronavirus disease, while the pandemic was ongoing. However, we did not want this new type of patient to jeopardize the results of our initial trial. On March 7^th the first SARS-CoV-2 infected patient was included in the Cape-Cod trial: as previously stated, all pre-specified selection criteria were satisfied. By March the 20^th 2020, twenty-six SARS-CoV-2 infected patients were randomized into the Cape Cod trial. We therefore decided not to wait anymore and started working on the challenging methodological issues associated with this unprecedented situation, an experience we want to share with the scientific community.

2. Methods

Challenges, trade-off and solutions

We involved a European Network of biostatisticians (Inspire and Asterix) to help us define the best way to proceed. Several methodological challenges had to be overcome: (i) should SARS-CoV-2 patients be part of a sub-group analysis or should we segregate SARS-CoV-2 patients from previously included patients, due to heterogeneity in disease characteristics and evolution, (ii) could we simultaneously include both SARS-CoV-2 and non-SARS-CoV-2 patients or should we interrupt the recruitment of non-SARS-CoV-2 patients while the pandemic is ongoing, (iii) the principal investigator wished to share the findings as promptly as possible - an efficient treatment would help manage future patients, an inefficient treatment should be abandoned quickly in order to explore alternative treatments, (iv) how could we share the results regarding SARS-CoV-2 patients without compromising the scientific rigor of the ongoing Cape-Cod trial, (v) relevant clinical major endpoints differed from the one used in the ongoing trial. We finally decided to embed a new trial, the Cape-Covid trial, within the Cape-Cod trial. This meant that (i) inclusions of non- SARS-CoV-2 infected patients had to be temporarily interrupted, (ii) the Cape-Covid trial would be conducted and (iii) once the Cape-Covid trial terminated, and after the pandemic had been contained, the Cape-Cod trial would be re-activated. We considered that doing so would allow us to resolve all the methodological issues listed above.

Regulatory, logistic and financial issues

The major practical point associated to Cape-Covid is that we were able to benefit from an existing legal and ethical framework, thus allowing the Cape-Cod trial to host the Cape-Covid trial. Regulatory and ethical approvals of necessary methodological adaptations were nevertheless mandatory, as well as complementary funding. Regulatory and ethical committees were fast and efficient in assessing our request for modifications. Additional funding was granted by the French Ministry of Health on March the 25^th 2020. On March the 31^st 2020, the ethics committee approved the major methodological changes that were propose: a primary outcome different from the one used in the parent Cape-Cod trial, a sequential design and the possibility of switching from a blinded trial toward an open one. Finally, on April the 9^th 2020, we obtained an approval from the French regulatory agency (Agence nationale de sécurité du médicament et des produits de santé – ANSM). The electronic Case Report Form (e-CRF) was updated. The Cape-Cod DSMB, initially composed of three members - an intensive care physician, a pulmonologist, and a statistician - was enlarged through the addition of an infectious diseases specialist, and a statistician with experience in group sequential trials. Biostatisticians often receive some training on group-sequential designs, but not many have experience of their use in practice. Moreover, as Cape-Cod trial was aiming at drug re-purposing in an institutional setting, the DSMB initial biostatistician had only basic knowledge on group-sequential methodology.

From a logistic point of view, we have chosen to make very few changes to the procedures related to the parent trial to take advantage of the experience gained by the centers. Similarly, we did not seek to open a new center in the midst of a pandemic. Ethics committees and regulatory agencies were given national instructions to conduct their assessments within a very short time frame. Information to the centers was given by e-mail and teleconference. All but 2 centers are members of the CRICS-TriGGERSep research network and are used to collaborative work and clinical trials in intensive care units.

Changes to the trial were updated on the ClinicalTrials.gov website, and publicly available on April the 15^th 2020. During this period, inclusion of eligible SARS-CoV-2 infected patients continued, within the regulatory and ethical framework of the Cape-Cod study.

Methodological adjustments

The Cape-Covid trial was planned with major changes, compared to the Cape-Cod trial. First, the primary outcome was defined as death or dependence on respiratory support at Day 21. Respiratory support includes mechanical ventilation or high-flow oxygen therapy. This new primary outcome makes sense at both the individual and societal levels. Indeed, prolonged respiratory support, especially mechanical ventilation, is associated with several complications (e.g. ventilator-associated pneumonia, ventilator-induced lung injury, acquired muscle weakness or sleep deprivation) and increased ICU length of stay. Moreover, in the context of a rapidly progressive pandemic with a risk of insufficient critical care capacities, diminishing the duration of respiratory support will decrease ICU length-of-stay and accelerate the liberation of ICU beds. Second, while the Cape-Cod trial is blinded, we considered the possibility of converting the Cape-Covid trial into an open label trial. Indeed, therapeutic units used to treat patients included in Cape-Covid will be those that were planned for use in the original Cape-Cod trial. However, when starting the Cape-Covid trial, only about 290 therapeutic units remained. Since obtaining new industry-manufactured placebo units seemed impossible during an ongoing pandemic, we decided (and therefore advised the ethics committee and the regulatory agency) that while the Cape-Covid trial was planned as being blinded, a conversion to an open label trial (or alternatively, to switch from a ready-to-use placebo to a placebo prepared immediately by a non-blinded research nurse) was possible if we needed to include more patients than we had therapeutic units. Third, the Cape-Covid trial was planned as a group sequential trial.

Embedding a group sequential study or subgroup analysis

When first considering the Cape-Cod trial, we have investigated the possibility of a subgroup analysis at the end of the trial for Covid-19 infected patients without stopping temporarily inclusion in Cape-Cod. However, several drawbacks have been raised; (i) the relevant clinical major endpoints were different between the two subgroups, (ii) as SARS-CoV-2 infection pathways were still investigated, we didn’t know if Community-Acquired Pneumonia patients could be secondarily also infected by SARS-CoV-2 virus, adding heterogeneity in the study, and (iii) waiting for the end of the Cape-Cod trial could take an additional year and there was a critical need for treatments against SARS-CoV-2 virus.

In the light of the above, we decided to embed the Cape-Covid study while temporally stopping the inclusion in Cape-Cod. Several group-sequential designs were investigated. Indeed, we needed to take into account on the one hand the feasibility in terms of software availability and fast training of the local team, and, on the other hand, the average expected sample size reduces due to interim analysis while the maximal calculated sample size increases. Although, we stayed conservative in terms of type I error rate, we had several exchanges about the pertinence of a conservative error rate in such fast spreading world-wide pandemic. One could argue that these are precious and timely data and that type I error rate control isn’t the most important point. However, the risk of selecting an inefficient treatment by increasing the type I error rate conducting to potentially treating thousands of patients before being able to spot it made the decision of keeping it at 5% level reasonable.

After investigating several group-sequential designs, as the Triangular test, Pocock’s and O’Brien and Fleming’s approaches (4-6), there were a common agreement to use a standard well-known method, the Lan-DeMets alpha-spending tests (7) (8), for its simplicity and for the continuous stopping boundaries that allow us to be more flexible in managing interim analysis. Indeed, since a large share of research personnel has been shifted to caring for patients, obtaining research data may be more difficult. One could also argue that there are more robust group-sequential designs available, but implementing these designs would have required time for training and investigations.

The need for available packages or functions in statistical software with fast computing times in order to study the sensitivity of the group sequential designs to the choice of stopping boundaries was of major importance (the time was very short before submitting the modifications to the protocol for approval).

We subsequently faced another important methodological issue. When calculating a sample size, we had a strong uncertainty about the failure rate in the control arm. Indeed, reported mortality rates of SARS-CoV-2 patients in ICU ranged from 26% to 73% (9) (10). However, severity assessment of patients was either absent or based on different criteria, and in some cases, patients still hospitalized in the ICU were unaccounted for. After a literature review and reaching out to several French ICUs (to have a better view of national mortality as health systems can differ from one country to another), we assumed a 30% rate in the control arm, considering the primary outcome, i.e., mortality or need of respiratory support at D21, for the control arm. The choice of this endpoint was debatable from a medical point of view at the time we needed to take this decision. We did not have sufficient evidence at the beginning of the pandemic to believe that any beneficial effect of corticosteroids would be of greater magnitude than in community-acquired pneumonia for which the Cape-Cod trial end-point was mortality at D28. At this stage of the pandemic, we feared that our intensive care capacities would be completely saturated. A treatment that could accelerate recovery and thus free up valuable beds more quickly seemed of major interest. On an individual level, prolonged stays in intensive care expose patients to an increased risk of complications and sequelae. This is why, in the particular context of the pandemic, we considered as a treatment failure whether the patient had died or was still present in intensive care at D21. Even if, retrospectively, as no other efficient treatment was meanwhile found, the need respiratory support at D21 is no longer a major concern.

Nevertheless, blinded sample size re-estimation could be necessary during the trial when updated knowledge about the control mortality rate becomes available. We planned to re-calculate the sample size if necessary.

Finally, having access to the methodological network support was very important especially when timely decisions were necessary (stopping inclusion in Cape-Cod study for patients’ safety, embedding the Cape-Covid study or not…) and there was a critical clinical/medical collective need to get treatments for patients. Being able to get a collegial methodological advice comfort us in our decisions.

Stopping boundaries

The trial was designed to demonstrate the superiority of the experimental treatment over placebo with an assumed event rate of 15% in the experimental arm with 80% power. This requires a sample size of 240 patients that was increased to 290, for 6 analyses under efficacy and futility boundary choices. We choose to keep a conservative boundary for efficacy and an aggressive boundary for futility. That is, stopping for high evidence of superiority while stopping early if the experimental treatment is not promising enough (therefore enabling subsequent patients to receive alternative treatments).

We investigated several futility boundaries (design details and Figure w1 are given in Supplementary File) under a simulation study using several possible scenarios under the null and the alternative hypothesis with asymmetric spending function with binding the lower bound. We decided to use a binding lower bound which implied to stop the trial if the boundary was crossed (which is not the case for non-binding bounds). It wasn’t ethical, in time of pandemic, to continue to administer patients an inefficient treatment, patients should be included in other studies if this situation occurs. In Table w1 (Supplementary file) are given the percentage of stopping at which interim analysis for several choices of Lan DeMets alpha-spending test boundaries and for each scenario. The choice of the futility boundary has a major influence at which interim analysis the trial will stop. It has less impact on the overall percentage for futility stopping (11).

The resulting boundaries for the trial are given in Figure 1.

Figure 1: Lan DeMets alpha-spending test stopping boundaries for the Cape-Covid trial

3. Conclusions

On June 15^th 2020, 149 SARS-CoV-2 infected patients had been included (including the 26 patients included before the regulatory and ethical approval of the Cape-Covid trial), and two interim analyses have been done. We have to acknowledge that we did not specify enough what could motivate a sample size review. Indeed, we didn’t know when published mortality rate in ICU will be reliable and stabilized. Therefore, we asked the DSMB members opinion after the second interim analysis. They did not consider useful to change anything. Otherwise, the number of cases has dramatically decreased. As a consequence, the DSMB agreed to have the third interim analysis realized with only 149 patients, a situation which can be coped with the approach used in this trial. Doing so will allow having this third interim analysis planned at the very end of June 2020, a period permitting to organize more easily a meeting of the DSMB members than during the summer months. We will then have to decide what should be done with this trial, knowing that there are very few new cases, a situation which also led, recently, the French authority (ANSM) interrogate sponsors of Covid-19 studies what they plan to do with their study. Finally, results from the RECOVERY trial which demonstrates that Dexamethasone induces a reduction in death were made publicly available on June 16th (12). These results may probably impact the standard of care, and question the relevance of continuing a placebo control trial as Cape-Covid. Recently, the Cape-Covid PI has been contacted by WHO to share the trial un-finished results for a meta-analysis of incomplete studies. If the PI decides to do so, this would imply to stop the trial as data on endpoint will be communicated.

We had to face a totally uncommon situation in which clinical research is part of the answer. As stated by Kalil et al (13), “by participating in an randomized clinical trial, both patients and clinicians can benefit from the unique opportunity to directly contribute to the discovery of new therapies, and also from the safer monitoring process in the conduct of clinical trials compared with uncontrolled drug administration (whereby safety cannot be determined)”.

Dean et al. (14) emphases the need to allow flexibility in study sample size, since fatality rates in therapeutic trials may need to be revised over time. They underscore the need for cooperation and coordination among research stakeholders, including funding agencies, which was the case here. Furthermore, Eichler et al. (15) emphasized the miss window of opportunities during the pandemic, indeed, there are many small stand-alone trials and observational studies of single-agent interventions which most will probably not deliver robust results that could support regulatory and patient-level treatment decisions. This is why international cooperation need to be supported by regulatory agencies, stakeholders and workshops. Our, shared experience has shown how an international methodological collaboration successfully elaborated a study design for the treatment of an unexpected disease within an existing clinical trial.

Nevertheless, we are still considering how to resume inclusions in the Cape-Cod trial after the pandemic. As described above, more than 830 patients have already been included over the last 3 years with 370 more need in order to reach sufficient power. If the Cape-Covid study doesn’t achieve the required sample size or is not terminated (for efficacy or futility) before next autumn, we will face possible inclusions of patients into two studies, as community-acquired pneumonia is a seasonal disease and Covid-19 will still be present. Patients’ heterogeneity should be addressed and methodological and practical solutions will have to be found.

Moreover, we presently have included half of the maximal required sample size, and recruitment has dramatically decreased over the last weeks. There is a possibility that we will not be able to include the required number of patients, and therefore to finish the study, a situation already observed during the Ebola outbreak (13). In this case, we will need to take contact with other principal investigators of studies evaluating corticosteroid (17 studies are evaluating corticosteroid in SARS-CoV-2 infected patients, source clinicaltrials.gov May 20^th 2020) in order to propose some meta-analysis of unfinished trials. This is unusual but the clinical question still needs to be answered.

In conclusion, we present a timely experience addressing the issue of how to urgently adjust an ongoing clinical trial which has unexpectedly started to include Covid 19 patients. In such a situation, it seemed to us that three priorities had to be respected: (1) despite the limited time available, obtain a reasonable consensus among the various stakeholders on ethical, logistical and methodological choices; (2) not lose sight of patient safety for a moment; (3) resist the rush by society and part of the scientific community and keep in mind to respond to the chosen issue as rigorously as possible. Some of our design choices could have been different if the situation wasn’t as critical as during this pandemic. Usually, we could have dedicated time and resource to investigate further design alternatives. Nevertheless, we still have much to learn about how to adapt, continue and re-start all clinical trials during and after the pandemic peak and doing so while keeping scientific integrity and preventing research being wasted.

Acknowledgements

Authors are grateful to Tim Friede, Martin Posh, Kit Roes, Nigel Stallard from the Inspire and Asterix European network for valuable discussions and insightful comments on the methodological options. We would also like to thank both John Whitehead and Thomas Jaki for their help on technical issues. We also would like to thank the DSMB members for their involvement: Jean Chastre, Béatrice Delasalle, Véronique Sébille, Sylvain Marchand-Adam, Liem Binh Luong.

Finally, we thank all trials stakeholders that continue scientific research even during a pandemic; Nicholas Heming, Ferhat Meziani, Gaëtan Plantefève, Marie Leclerc, Carine Coffre, Hélène Bourgouin, Céline Lengellé, Caroline Caille-Fénérol, Guillaume Voiriot, Julio Badié, Bruno François, Cécile Aubron, Stephan Ehrmann, Djillali Annane and the CRICS-TRIGGERSEP Network.

Inspire: https://warwick.ac.uk/fac/sci/med/research/hscience/stats/completedprojects/inspire/

Aterix: http://www.asterix-fp7.eu/

TRIGGERSEP Network: https://www.triggersep.org/

Contributors

BG SZ and PFD wrote the first draft. All authors critically revised the draft for important intellectual content, and gave final approval of the version to be published. BG is the guarantor for the study.