Marginal effects and significance testing with Heckman's sample selection model: a methodological note

Abstract

This article illustrates two techniques for calculating the statistical significance of the marginal effects derived from Heckman's sample selection model, an increasingly common econometric specification in economics and political science. The discussion draws on an analysis by Sweeney (2003) of the incidence and intensity of interstate disputes. After replicating his results, the article presents the delta method and the nonparametric bootstrap as alternative techniques for obtaining SEs of the marginal effects, which themselves are calculated from a transformation of the model parameters. The analysis reveals two variables for which misleading inferences are drawn with respect to the precision of the estimated coefficients in the original study, suggesting that significance testing of the derived marginal effects is warranted.

Acknowledgements

The author wishes to express gratitude to Kevin Sweeney for providing the data and for helpful insights on the implementation of the Heckman model, and to Manuel Frondel, Christoph Schmidt and Matthias Sinning, for their comments on an earlier draft.

Notes

¹ In an excellent overview of censored regression models, Dow and Norton (2003) suggest an alternative formula for measuring what they refer to as the actual outcome, which they contrast with the potential outcome given by Equation 1(1). The actual outcome refers to that which is observed, whereby zeros in the data are treated as true zeros rather than as missing values. The potential outcome, by contrast, refers to a partially observed latent variable that treats zeros in the data as censored. The statistical tests discussed here can be easily extended to the analysis of marginal effects corresponding to the actual outcome.

² The code for implementing the techniques, written in Stata, is available from the author upon request. The data and original code used by Sweeney can be downloaded from the website of The Journal of Conflict Resolution, http://jcr.sagepub.com/

³ He is, however, cautious in interpreting his finding of significance, noting that the Z-statistic obtained from his analysis of the sample variation in the marginal effect is insignificant.