Effects of using specific versus general data in social program research

Abstract

We present a comparison of results of similar analyses of a particular social program using two sources of data: one representative of the general population, and one representative of the population that actually is eligible for the social program. To do this, we focus on a particular public assistance program as implemented in Florida as our public policy program of choice. We compare the results of an analysis of the program's participation rates using US Census data for the general population of Florida to the results of the exact same model using a dataset that includes only the population of Florida that is actually eligible for the program. We find that while generally signs of effects remain the same, they do not always remain the same; moreover, significance differs, and the marginal effects of various demographic and socio-economic factors on program participation rates vary greatly. We submit that such differences are important for policymakers to recognize so that they can effectively target programs to those individuals and geographic areas most in need of such programs.

Notes

¹ Owing to the manner in which the state of California enrols Lifeline subscribers, more than 100% of eligible households enrolled in the program, i.e. some enrolled despite not meeting eligibility criteria.

² Our formal analysis of the Lifeline Assistance Program in Florida is forthcoming in Information Economics and Policy; our analysis of the Lifeline Assistance Program nationwide is in Public Finance Review.

³ During the time period of this study (2003–2005), some bills of Florida's Lifeline customers show Lifeline credits to be in excess of $13.50. As part of the telecommunications price rebalancing in Florida, BellSouth (recently merged with AT&T), Sprint (now called Embarq), and Verizon were restricted by statute from increasing local telephone prices for Lifeline customers. BellSouth implemented this restriction by providing a credit on the bills for its Lifeline customers equal to the increase in local telephone prices that occurred with BellSouth's price rebalancing. Instead of providing a new credit on their customers’ bills, Sprint and Verizon increased the Lifeline credit on the bills of their Lifeline customers, so the Lifeline credits to their customers’ bills appear to exceed $13.50.

⁴ Holt and Jamison (2006) provide a history of Florida's eligibility criteria.

⁵ The Universal Service Fund (USF) is an FCC program amended by the Telecommunications Act of 1996 to provide various levels of support to programs designed to ensure all US citizens have affordable access to basic telecommunications service. The USF covers four programs: high-cost support for incumbents operating in financially uneconomic areas; Lifeline and Link-Up to reduce low-income consumers’ basic telephone expenses; schools and libraries to reimburse telecommunications providers for discounts on services provided to schools and libraries; and rural health to provide discounts to rural health care providers for telecommunications and Internet services.

⁶ Link-Up provides a one-time subsidy to low-income households to help initiate telephone service.

⁷ This model is from the paper ‘Participation in Social Programs by Consumers and Companies: A Nationwide Analysis of Participation Rates for Telephone Lifeline Programs’ by Hauge et al. (2007b) in Public Finance Review.

⁸ We expect stigma costs to be low given studies that have shown stigma decreases with program participation and the primary manner of proving eligibility for Lifeline is through proving participation in another welfare program. Furthermore, a household can enrol from the privacy of their home, avoiding any stigma associated with visiting a welfare office. In addition to stigma, the Lifeline program carries some effort cost of enrolling. In Florida, most enrolment is accomplished through checking a box on a form received upon receipt of any other qualifying welfare service. Additionally, applicants can request Lifeline online through the Department of Children and Families (DCF).

⁹ Surveys conducted by the Public Utility Research Center at the University of Florida reveal a lack of information about the program to be a main reason for nonparticipation. Surveys are available at www.purc.ufl.edu. The surveys include interviews of Floridians in person and over the telephone, as well as written surveys of households that qualify but do not participate and those that qualify and had disconnected their telephone service.

¹⁰ The number of Lifeline participants is provided by each company at the city level. We aggregate to the county level to correspond to the level of the eligible household data.

¹¹ The weights are 1/[n_ip_i (1−p_i )], where n_i is the total eligible households of county i and p_i is the logit probability of the Lifeline participation rate in county i. As discussed by Greene (2003), p_i must be estimated since it is a function of unknown parameters. Following the prescribed procedure, we estimate p_i using Ordinary Least Squares (OLS) in a first stage because all that is needed is a consistent estimate of p_i . The weights are then computed and used in a second-stage FGLS estimation. Several specifications of the dependent variable are available. We choose the minimum logit chi-square estimator due to its similarity to the well-known logit dichotomous choice model. To test robustness, we estimated the model using another commonly employed specification in which the dependent variable is simply the log of the participation rate, finding no major differences between the two specifications. There are 13-year county observations with zero subscribers. Rather than delete these observations, the number of subscribers is arbitrarily assigned to one in order to compute the dependent variable. Although clearly ad hoc, this solution is the most viable in our situation. We also include a dummy variable in our estimation to account for the 13 observations; the dummy variable (not reported for brevity) is significantly negative.

¹² Some county-level Lifeline participation data are available back to 2000; however, prior to 2003 data are incomplete. We choose to analyse the most recent 3 years since they represent complete panels. Summary statistics for 2000–2005 are available from the authors.

¹³ A Hausman test indicates that the assumption of the random effects model concerning the orthogonality of the random effects and the regressors is appropriate. The chi-square statistic (15 d.f.) is 20.88, which is insignificant at any conventional level. Thus, we cannot reject the null of no correlation between the random effects and the regressors. Complete results of this test are available from the authors. We choose the random effects model because it allows the inclusion of time-invariant regressors and is more efficient than a fixed-effects model.

¹⁴ Notes on this framework are attributed to Damon Clark, Assistant Professor of Economics at the University of Florida.

¹⁵ The formal proof is in the Appendix.

¹⁶ Hauge et al. (2007a) Discounting telephone service: an examination of participation in the Lifeline assistance program using panel data, Working Paper No. 06-06, University of Florida, Public Utility Research Center, forthcoming in Information Economics and Policy.

¹⁷ Holt and Jamison (2006). Making telephone service affordable for low-income households: an analysis of Lifeline and Link-up telephone programs in Florida, Working Paper No. 06-05, University of Florida, Public Utility Research Center. The full paper is available at http://www.purc.ufl.edu/Lifeline2.htm.

¹⁸ Williamson worked with various agencies as well as undertaking her own statistical calculations. For example, the DCF supplied participation data for the TANF, Medicaid and Food Stamp programs for the years 2003 through 2005. These data were given by individual recipients; therefore, a case study number (typically applied by household) and further calculations were used to estimate eligible households. Additionally, data for recipients of Section 8 Housing assistance from 2001 through 2005 were compiled using primary data provided by Housing and Urban Development (HUD). Details of the methodology for estimating the total number of households and various characteristics of heads of household are provided in Williamson (2006).

¹⁹ Results are available from the authors upon request.

²⁰ The marginal effects are simulated by changing the relevant independent variable, recomputing the predicted rate for each individual, and comparing this new prediction to the rate predicted from the original sample.

²¹ Garbacz and Thompson (2005) suggest that in developing countries, universal service might be promoted more effectively with subsidies for cellular telephones.

²² The manner by which the local telephone rate might impact household participation decisions remains an unresolved question. We suggest that if a poorer household is more likely to sign up for any welfare program, and an increase in the local telephone rate makes the household poorer, the reported relationship would hold. This would require an income effect off-setting the substitution effect. It is beyond the scope of this article to examine these effects and the associated demand elasticities of households of different income levels, however, we believe it an important question for further research.

²³ Interestingly, a 2004 paper by Kang et al. analysing TANF found that an increase in nonlabour income decreases welfare participation. For our study, this would suggest that participation in TANF, Food Stamps or SSI, for example, would decrease participation in Lifeline. Further research into this possibility seems warranted.

²⁴ In 2003, there were 12 402 000 single parents in the United States. 81.78% (10 142 000) were single mothers (US Census, 2004).

²⁵ This proof is attributed to Damon Clark, University of Florida, Assistant Professor of Economics.

²⁶ Formal proof is available upon request.