Evaluating the Impacts of an Enhanced Family Self-Sufficiency Program

Abstract

We conduct an impact analysis of the Denver, Colorado, Housing Authority’s Home Ownership Program (HOP) employing quasi experimental methodologies (i.e., nearest-neighbor matching, inverse probability weighting with regression adjustment) that permit causal inferences of program impacts with substantial confidence. HOP is an unusual, enhanced variant of the Family Self-Sufficiency program that incentivizes and assists participants’ purchase of a home. We analyze whether, compared with the control group, HOP participants exhibited significantly greater earnings growth during the program, enhanced economic security, and rates of home buying. We find that participants with a high intensity of treatment showed significant improvement in all outcomes. Results are robust to model specification and insensitive to omitted variable bias typically found in the social sciences. We conclude that a well-conceived and well-executed public housing authority program aimed at building the financial, human and social assets of low-income households receiving housing assistance can yield substantial benefits to participants.

Keywords:

• Financial capability
• asset development
• family self-sufficiency
• quasi-experimental methods

Acknowledgments

This research was supported by grants from the Ford Foundation and the John D. and Catherine T. MacArthur Foundation. The opinions expressed herein do not necessarily reflect those of our funders. The authors wish to thank Renee Nicolosi of the Denver Housing Authority for providing administrative data and unstinting support for our research. Three anonymous referees offered valuable suggestions for clarifications and extensions of an earlier draft.

Notes

1. Regrettably, there have been relatively low graduation rates for those who have enrolled in DHA’s FSS and HOP programs since 1995: 25% and 20%, respectively.

2. We excluded 265 enrollees from the sample used to statistically model program impacts because of inability to ascertain the HOP beginning or end date information needed to estimate intensity of treatment. Further, for enrollees who entered HOP in 2008 and 2009 and who did not have a reported end date or a current participant flag in the administrative data, we assumed that they were still active in the program since in any given year approximately 400 residents participate in HOP.

3. Requirements for entry into the Home Buyers Club include being employed with the current employer for at least one year (or having another stable source of mortgage repayments) and having personal savings of at least $500.

4. Home Buyers Club participants earn $20 toward downpayment and/or closing costs for every such class they attend.

5. Our matching strategy focused on participant characteristics that were identified in our earlier work as fundamental in the decision to enroll in HOP (Santiago & Galster, 2001). We note that the age and disability status of household head and family size at time of HOP program entry are comparable between our treatment and control groups. As noted earlier, being employed or having an outside income are not criteria for initial HOP participation. Household income at the time of program entry is skewed with a high proportion of enrollees (20%) reporting $0 income; median family income was $9,438 for our control group and $14,428 for our treatment group. Unfortunately, we do not have measures of income or other socioeconomic indicators prior to HOP program entry. We would argue that matching on participation in FSS addresses concerns about selection based on income inasmuch as participation in FSS is predicated on the need to enhance employment and earnings. Matching covariate balance tables are available upon request.

6. Preliminary analyses did not find significant differences in outcomes between low-intensity and moderate-intensity participants, hence our decision to combine these two groups into one control group.

7. As noted by Santiago and Galster (2004), self-efficacy reflects a nine-item measure of belief in the ability to complete a task, in this case improving financial capability and completing the homeownership program.

8. Estimate of ATET is conducted in Stata 14 using teffects nnmatch with robust Abadie and Imbens (2011) standard errors. Covariate balance tables are available upon request. We use the rbounds package in Stata 14 to conduct sensitivity analysis for continuous variables (i.e., annual income change during HOP) and the mhbounds package (Becker & Caliendo, 2007) for the two binary outcomes (i.e., economic security exit and purchased home within five years of HOP start). We tested hypothetical odds ratios of omitted variables from 1 to 10 in 0.1 increments. Because these methods assume matching with replacement, the sensitivity analysis is based on an estimate of ATET using matching with replacement in psmatch2 in Stata.

9. For simplicity we present the impact estimates derived from the test of the high-intensity group compared with a matched sample of the combined low- and moderate-intensity group.

10. Q mh = ( |Y₁ − ∑^S_{s = 1} E(Y_1s)| − 0.5) / (∑^S_{s = 1} Var(Y_1s))⁻² where Y₁ is the number of participants, E(Y_1s) is the expectation of successes in stratum s, and Var(Y_1s) is the variance of successes in stratum s. See Becker and Caliendo (2007) for more detail.

11. We did not ask about expected earnings so we do not have a direct comparison.