Generalizing Clinical Trial Results to a Target Population

Abstract

The external validity of the estimated treatment effect from a clinical trial is in doubt when there are effect modifiers whose distribution in the target population differs from that in the trial. Adjusting an estimated treatment effect from a trial to predict its likely value for the target population is a process known as generalization. We review classical and contemporary approaches to this problem. The traditional method is post-stratification, or the reweighting of stratum-specific treatment effect estimates by population distribution proportions. Contemporary methods employ stratification or weighting techniques based on estimates of the probability that an individual is included in the trial, akin to the propensity score. These methods are more flexible in that they readily accommodate continuous effect modifiers. Estimating the probabilities, however, requires in principle that one have individual-level population data, which are seldom available for pharmaceutical trials. When the effect modifiers are all discrete, the post-stratification and probability-weighting approaches give essentially the same estimates. Naïvely computed standard errors with the contemporary methods may be inflated. We illustrate and compare generalization methods in a simulation and using data from the Lipids Research Clinics Coronary Primary Prevention Trial and the New York School Choice Experiment.

Keywords:

• Causal inference
• Generalizability
• Post-stratification
• Propensity score
• Randomized clinical trial

Acknowledgments

We obtained the LRC-CPPT data from the NHLBI Biologic Specimen & Data Repository Information Coordinating Center. Our work does not necessarily reflect the opinions or views of the LRC-CPPT or the NHLBI. We thank John Barnard and Donald Rubin for providing the New York School Choice dataset. This work is part of the first author’s doctoral dissertation under the supervision of the second author in the Department of Statistical Science at Southern Methodist University.