A framework to assess the value of subgroup analyses when the overall treatment effect is significant

ABSTRACT

Although subgroup analysis has been developed and widely used for many years, it is still not clear whether we should perform and how to perform such subgroup analyses when the overall treatment effect is significant. In this paper, we develop a framework to assess and compute the long-term impact of different strategies to perform subgroup analysis. We propose two performance measures: the average gain for patients in the future (E) and the probability of recommending a change to a worse treatment at individual patient level (P). Five families of decision rules are applied under different assumptions for the individual treatment effect (TE) variation. Three distributions reflecting optimistic, moderate, and pessimistic scenarios are assumed for true treatment effects across studies. This framework allows us to compare subgroup analyses decision rules, and we demonstrate through simulation studies that there are decision rules for subgroup analysis which can decrease P and increase E simultaneously compared to the situation of no subgroup analysis. These rules are much more liberal than the usual superiority testing. The latter typically implies a dramatic decrease in E.

KEYWORDS:

• Confidence interval
• decision rules
• overall treatment effect
• subgroup analysis
• subgroup effect

Acknowledgments

The authors are grateful to Frank Bretz, Marc Buyse, Martin Schumacher, and two referees for providing valuable and helpful comments. We also thank Kim Harris for English editing.

Funding

This work was supported by funding from the European Community’s Seventh Framework Programme FP7/2011: Marie Curie Initial Training Network MEDIASRES (“Novel Statistical Methodology for Diagnostic/Prognostic and Therapeutic Studies and Systematic Reviews”; www.mediasres-itn.eu) with the Grant Agreement Number 290025.

Supplemental Material

Supplemental data for this article can be accessed on the publisher’s website.

Additional information

Funding

This work was supported by funding from the European Community’s Seventh Framework Programme FP7/2011: Marie Curie Initial Training Network MEDIASRES (“Novel Statistical Methodology for Diagnostic/Prognostic and Therapeutic Studies and Systematic Reviews”; www.mediasres-itn.eu) with the Grant Agreement Number 290025.