Reanalysis of the Harvard Six Cities Study, Part I: Validation and Replication

Abstract

Because the results of the Harvard Six Cities Study played a critical role in the establishment of the current U.S. ambient air quality objective for fine particles (PM_2.5), the U.S. Environmental Protection Agency, industry, and nongovernmental organizations called for an independent reanalysis of this study to validate the original findings reported by Dockery and colleagues in the New England Journal of Medicine (vol. 329, pp. 1753–1759) in 1993. Validation of the original findings was accomplished by a detailed statistical audit and replication of original results. With the exception of occupational exposure to dust (14 discrepancies of 249 questionnaires located for evaluation) and fumes (15/249), date of death (2/250), and cause of death (2/250), the audit identified no discrepancies between the original questionnaires and death certificates in the audit sample and the analytic file used by the original investigators. The data quality audit identified a computer programming problem that had resulted in early censorship in 5 of the 6 cities, which resulted in the loss of approximately 1% of the reported person-years of follow-up; the reanalysis team updated the Six Cities cohort to include the missing person-years of observation, resulting in the addition of 928 person-years of observation and 14 deaths. The reanalysis team was able to reproduce virtually all of the original numerical results, including the 26% increase in all-cause mortality in the most polluted city (Stubenville, OH) as compared to the least polluted city (Portage, WI). The audit and validation of the Harvard Six Cities Study conducted by the reanalysis team generally confirmed the quality of the data and the numerical results reported by the original investigators. The discrepancies noted during the audit were not of epidemiologic importance, and did not substantively alter the original risk estimates associated with particulate air pollution, nor the main conclusions reached by the original investigators.