Are Education Cost Functions Ready for Prime Time? An Examination of Their Validity and Reliability

Abstract

This article makes the case that cost functions are the best available methodology for ensuring consistency between a state's educational accountability system and its education finance system. Because they are based on historical data and well-known statistical methods, cost functions are a particularly flexible and low-cost way to forecast what each school district must spend to meet the standards in a state's accountability system. However, the application of cost functions to education must confront several challenges in both data collection and estimation methodology. This article describes the strengths and weaknesses of various ways to address these challenges and illustrates how the reliability and forecasting accuracy of cost functions can be tested using data for Missouri school districts.

Notes

Education cost functions have been estimated for Arizona (Downes & Pogue, 1994), California (Duncombe, Lukemeyer, & Yinger, 2008; Imazeki, 2008), Illinois (Imazeki, 2001), New York (Duncombe et al., 2008; Duncombe & Yinger, 2000, 2005b), Texas (Gronberg, Jansen, Taylor, & Booker, 2004; Imazeki & Reschovsky, 2004a, 2004c), and Wisconsin (Reschovsky & Imazeki, 2001).

For a derivation, see Henderson and Quandt (1980, p. 85).

In their studies of Texas, Imazeki and Reschovsky (2004a, 2004c) and Gronberg et al. (2004) used measures of value-added across 1 or 2 years.

Ruggiero (1998) showed how to separate cost and efficiency factors in DEA, but his approach requires far more observations than are available for any state because each district must be compared with other districts that have the same performance and the same cost factors. A multistage DEA-based approach has been used by McCarty and Yaisawarng (1993), Ray (1991), and Ruggiero (2001). Another approach is a stochastic frontier regression (Alexander et al., 2000; Gronberg et al., 2004). Ondrich and Ruggiero (2001) showed, however, that stochastic frontier regression produces the same results as an OLS regression except that the intercept has been shifted up to the frontier. As a result, this approach does not remove biases caused by omitting controls for efficiency.

In a cost-function context, it is not possible to separate inefficiency associated with “wasteful” spending from inefficiency associated with spending on performance measures other than those included in S. It follows that a given school district could be deemed inefficient in providing one measure of student performance, say, math and English scores, and efficient in providing another, say, art and music.

Scholars face a trade-off with this approach. If the “similar” districts are literally neighbors, then their exogenous characteristics might be correlated with unobservable sorting factors shared with the district that defines the observation, thereby creating an endogeneity problem. If the “similar” districts are too far away geographically, however, they may not be part of the district's comparison group so that their traits have no explanatory power in the first stage regression. In this article we compromise between these two extremes by selecting the traits of districts in the same labor market area.

Most studies use a variant of the Cobb-Douglas function, which is multiplicative in form. The Cobb-Douglas function assumes that the elasticity of substitution between all inputs is equal to one and that the elasticity for economies of scale is constant at all levels of output.

One of the most popular flexible cost functions used in empirical research is the translog cost function. A translog cost model includes squared terms for each input price and outcome, and adds interaction terms between all factor prices, and outcomes. Gronberg et al. (2004) also included a number of interaction terms between outcomes, teacher salaries, and nonschool factors. In all, they have more than 100 variables in their cost function for Texas compared to 18 variables in the Texas cost model estimated by Imazeki and Reschovsky (2004a).

On this point we are at least in agreement with Loeb (2007), who said that “the primary drawback of this method,” which she called the “evidence-based” approach, “is that the research base is not strong enough to support it” (p. 13).

Imazeki (2008) made this argument, too. She claimed that “if the data and model were perfect (i.e., correctly specified with no unobservable variables or measurement error), the final cost estimates from the cost function and production function should be similar” (p. 102). This is simply not correct. As discussed next, spending and inputs are not the same thing and there is no reason at all to expect the final cost estimates from a misspecified production function to yield cost estimates that are similar to those from a cost function.

In addition, we find it ironic that CHL, who are so critical of the treatment of efficiency in cost function studies, do not find fault with the production function approach, which faces much more severe challenges in dealing with efficiency.

Data for approximately five school districts are not available. See Duncombe (2007) for a more in-depth discussion of the variables used in this cost model.

Special education services in St. Louis County and Pemiscot County are provided by special school districts serving these counties. Total spending, counts for special education students, and counts of students receiving subsidized lunch in these two special school districts are assigned to the regular school districts in each county using the share of county enrollment in each regular school district. For example, if a regular school district had 10% of St. Louis County enrollment, then it would be assigned 10% of the spending, 10% of special education students, and 10% of the subsidized lunch students in the special district serving St. Louis County.

K8 students attending only one K12 district are assigned the high school proficiency rates for math and communication arts in this K12 district. In a few cases students in a K8 district attended two K12 districts for high school. To assign a high school performance measure to a K8 district, we constructed a weighted average of proficiency for high school math and communication arts exams, where the weight is based on relative enrollment. For example, assume students in the K8 district A attended K12 Districts B and C, where the enrollment in District B is 6,000, and enrollment in District C is 4,000. Then the high school performance assigned to District A is based on a weighted average of high school performance in Districts B and C, where the weights are 60% and 40%, respectively.

Although other professional staff are key resources as well, variation in other professional salaries across districts are typically highly related to variation in teacher salaries (correlation over 0.75). We include only teacher salaries in the cost model.

To control for variation in education and experience across districts, the natural logarithm of teacher salaries is regressed on the log of total experience, and an indicator variable for whether the teacher has a graduate degree. We use the regression to estimate average salaries for teachers in each district with the statewide average experience (between 0 and 5 years) and the statewide average percentage of teachers with a graduate degree.

In other cost function estimates we have used a series of dummy variables to capture different enrollment size categories. Although this is a flexible way of measuring the relationship between spending and enrollment, the forecasting accuracy of models using these variables was slightly worse than using a quadratic relationship for enrollment. We also checked forecasting accuracy when a cubic term is included in the model and did not find it improved accuracy (and it was statistically insignificant).

Another measure of child poverty is the child poverty rate produced by the Census Bureau every 10 years as part of the Census of Population. Although this measure is updated on a biennial basis, the updates are based on the decennial Census estimates, which implies that they may be quite inaccurate by the end of every decade. We found that the subsidized lunch rate in 2000 had a correlation of over 0.7 with the Census child poverty rate.

We also tried interactions of the subsidized lunch rate with pupil density, enrollment, and percent college educated adults. Only the interaction with percentage African American and with pupil density were statistically significant. When a quadratic specification for subsidized lunch was tried, the coefficient on the squared term was not significant.

Unlike subsidized lunch, there are no federal standards on how LEP students are measured, and typically no auditing process to assure that the data are accurate. In Missouri, student language data are collected in the Limited English Proficient Student Census (or English Language Learners Census) in October of each year. To evaluate the accuracy of the LEP data collected by Missouri, we compared this data to an alternative measure available in the 2000 Census of Population—the percentage of students, who live in a household where English is not spoken well at home. The LEP measure supplied by school districts in Missouri is not highly correlated (r = .30) with the Census measure, suggesting that there are inconsistencies in how districts are classifying and reporting LEP students.

The income data lags several years, so that the income data from the 2002 calendar year are used for the 2004–05 school year.

The state aid measure includes minimum guarantee aid (basic formula) and aid for free and reduced price lunch students.

*indicates a coefficient significantly different from zero at the 5% level.

**indicates a coefficient significantly different from zero at the 10% level.

^aExpressed as a natural logarithm.

*indicates a coefficient significantly different from zero at the 5% level.

**indicates a coefficient significantly different from zero at the 10% level.

^aExpressed as a natural logarithm.

^bLow ratio is a ratio below 0.05 and high ratio is above 0.2.

The model was estimated with xtivreg2 in STATA (Schaffer, 2005). Another weak instrument test involves comparing Kleibergen-Paap rk statistic to critical values established by Stock and Yogo (2005). Although this comparison is not technically correct given non-i.i.d errors, Baum, Schaffer, and Stillman (2007) argued that this is a reasonable approximation. The Kleibergen-Paap rk statistic is generally below the critical values established by Stock and Yogo (2005) for 10% relative bias. In other words, this test would suggest the potential for weak instruments in both Models 1 and 2.

For each variable a district can influence (outcome measure and efficiency-related variables), the estimated coefficient of the cost model is multiplied by some constant, typically the state average for that variable. For each cost factor outside of district control, the estimated coefficient from the cost model is multiplied by the actual values for the district. The sum of the products for factors outside and within district control is used to predict costs in a district with average outcomes and efficiency. Predicted costs are also calculated for a hypothetical district, which has average values for all variables in the cost model. Predicted spending in each district is divided by spending in this average district (and multiplied by 100) to get the overall cost index.

Another decision is whether to keep the 1st year of the model fit period fixed (rolling origin evaluations) or to keep the period used to estimate the forecasting model fixed (rolling window evaluations). Rolling origin evaluations use the maximum data available to fit the model, whereas rolling window evaluations create a more equal comparison across different forecasts (Tashman, 2000). We use a rolling window forecast based on 3 years of data for this article.

The average absolute PE has been criticized as having a bias favoring underestimates compared to overestimates (Armstrong, 1985). An alternative is to divide the error by the average of the forecast and actual values rather that the actual values. Armstrong (1985) called this the adjusted mean absolute PE.

The Theil U-statistic is commonly measured as the square root of the ratio of the square of the PE divided by the square of the PE for the naïve forecast. The PE of the naïve forecast is actual value in this period minus the actual value for the last period divided by the actual value for this period. If the Theil U-statistic is below 1, then the forecast does better than the naïve forecast. See Collopy and Armstrong (2000) for a discussion of alternative versions of the Theil U-statistic.

Per-pupil spending, student performance and teacher salaries are logged. The fit of the model was weak (adjusted R ² = .064).

*indicates a coefficient significantly different from zero at the 5% level.

**indicates a coefficient significantly different from zero at the 10% level.

^aExpressed as a natural logarithm.

^bLow ratio is a ratio below 0.05 and high ratio is above 0.2.