Sensitivity analysis of the unconfoundedness assumption with an application to an evaluation of college choice effects on earnings

Abstract

We evaluate the effects of college choice on earnings using Swedish register databases. This case study is used to motivate the introduction of a novel procedure to analyse the sensitivity of such an observational study to the assumption made that there are no unobserved confounders – variables affecting both college choice and earnings. This assumption is not testable without further information, and should be considered an approximation of reality. To perform a sensitivity analysis, we measure the departure from the unconfoundedness assumption with the correlation between college choice and earnings when conditioning on observed covariates. The use of a correlation as a measure of dependence allows us to propose a standardised procedure by advocating the use of a fixed value for the correlation, typically 1% or 5%, when checking the sensitivity of an evaluation study. A correlation coefficient is, moreover, intuitive to most empirical scientists, which makes the results of our sensitivity analysis easier to communicate than those of previously proposed methods. In our evaluation of the effects of college choice on earnings, the significantly positive effect obtained could not be questioned by a sensitivity analysis allowing for unobserved confounders inducing at most 5% correlation between college choice and earnings.

Keywords:

• causal inference
• effects of education
• propensity score
• register data

Acknowledgements

We are grateful to Per Johansson, Maria Karlsson, Sune Karlsson and Thomas Laitila for their helpful comments on an earlier version of this paper, and to Kent Eliasson and Olle Westerlund for their discussions on the effects of college choice on the labour market. The Swedish Council for Working Life and Social Research, the Institute for Labour Market Policy Evaluation and the Swedish Research Council through the Ageing and Living Condition program at Umeå\ University funded the research presented in this paper. We acknowledge the Swedish Agency for Growth Policy Analysis (formerly the Swedish Institute for Growth Policy Studies) for providing access to the register data used in this paper.