A Cautionary Tale of Evaluating Identifying Assumptions: Did Reality TV Really Cause a Decline in Teenage Childbearing?

ABSTRACT

Evaluating policy changes that occur everywhere at the same time is difficult because of the lack of a clear counterfactual. Hoping to address this problem, researchers often proxy for differential exposure using some observed characteristic in the pretreatment period. As a cautionary tale of how difficult identification is in such settings, we re-examine the results of an influential paper by Melissa Kearney and Phillip Levine, who found that the MTV program 16 and Pregnant had a substantial impact on teen birth rates. In what amounts to a difference-in-differences approach, they use the pretreatment levels of MTV viewership across media markets as an instrument. We show that controlling for differential time trends in birth rates by a market's pretreatment racial/ethnic composition or unemployment rate causes Kearney and Levine's results to disappear, invalidating the parallel trends assumption necessary for a causal interpretation. Extending the pretreatment period and estimating placebo tests, we find evidence of an “effect” long before 16 and Pregnant started broadcasting. Our results highlight the difficulty of drawing causal inferences from national point-in-time policy changes.

KEY WORDS:

• Difference-in-differences
• Shift-share instruments
• Social media
• Teen pregnancy.

Acknowledgments

The authors thank Phil Levine for his generous assistance in replicating results, and Rajeev Dehejia, Bill Evans, Gary Solon, two anonymous referees, and seminar participants at Columbia University, the CUNY Graduate Center, the CUNY Institute for Demographic Research, the Guttmacher Institute, Princeton University, and the University of Michigan for helpful comments. Onur Altindag and J. J. Chen provided exemplary research assistance. Joyce and Kaestner acknowledge support from the National Institute of Child Health and Human Development Grant R01 HD082133 and from the Health Economics Program of the National Bureau of Economic Research. Nielsen ratings data are proprietary and cannot be shared. Birth data from the National Center for Health Statistics (NCHS) have confidential elements and must be obtained directly from NCHS. Institutional Review Board (IRB) approval was obtained from the National Bureau of Economic Research.