Using Propensity Scores in Educational Research: General Principles and Practical Applications

ABSTRACT

Educational researchers frequently study the impact of treatments or interventions on educational outcomes. However, when observational or quasiexperimental data are used for such investigations, selection bias can adversely impact researchers’ abilities to make causal inferences about treatment effects. One way to deal with selection bias is to use propensity score methods. The authors introduce educational researchers to the general principles underlying propensity score methods, describe 2 practical applications of these methods, and discuss their limitations.

Keywords:

• observational studies
• propensity scores
• quantitative methods

Notes

1. It is also important to note that mathematics teachers were not randomly assigned to problem-solving emphasis levels in their classes. However, given that intact classes were not studied, and, in most cases, there were very few LSAY students in any particular mathematics class, in this example we focused on selection bias due to observable differences in students between the treatment and control groups. A more sophisticated analysis could also consider the impact of selection bias due to teacher characteristics. See Hong and Raudenbush (2005, 2006) for an example of propensity score methods applied to a multilevel design.

2. Note that by construction of each block defined by this procedure, the covariates are balanced and the assignment to treatment can be considered random. In other words, for a given propensity score, treated and control units should be on average observationally identical (Becker & Ichino, 2002).