Maximizing the Usefulness and Use of Large Federal School-Data Collections

Abstract

This article explores ways to improve the use and usefulness of four large and independently developed U.S. Education Department (ED) school-based data collections: the annual EDFacts’ student outcomes on state assessments and high school graduation rates; the annual Common Core of Data’s student and teacher counts and district and state finances; the biennial Civil Rights Data Collection’s topics on access and barriers to opportunity; and the biennial National Teacher and Principal Survey’s data on demographics, salaries, training, and opinions. Three data sets cover the universe of K–12 schools and districts. To make these collections more useful, the paper proposes combining these separate data sets at school, district and state levels; linking collections into longitudinal/time series files, improving timeliness and accuracy, and developing a single portal for online access with a comprehensive data tool (e.g., NAEP Data Explorer). The new data system would greatly increase scholars and practitioners capacity to examine and explore relationships between outcomes and inputs over time. The paper suggests expanding data uses including for building education indicators, benchmarking, and evaluations; supporting nonfederal users; and building a report bank.

Keywords:

• more useful data
• better data explorers
• K–12 data
• combining big data sets

Acknowledgments

The authors wish to thank an extraordinary set of reviewers for their substantive comments that expanded the scope of the databases we examined, the limitations we identified with the current school-data collections and data tools, and the strategies that we recommended should be addressed to improve data usefulness and use.

Notes

1 The four data collections are: Common Core of Data (CCD) (2019); EDFacts Initiative (2018); Civil Rights Data Collection (2019); and National Teacher and Principal Survey (NTPS, 2018).

2 Office for Civil Rights (OCR)-protected groups refer to race/ethnicity, gender, disability, and English–language learner status. Low-income students and migrants are student groups EDFacts covers, but CRDC does not.

3 The NTPS data files are only available to users who have a federally approved restricted-use license.

4 The annual dollar cost of each survey is estimated as (1) EDFacts: value of 126,880 annual burden hours including CCD (2018) at $50 per hour is $6.3m; (2) CRDC: value of 761,000 annual burden hours is $38m annual costs plus $3.9m annual data processing totaling about $42 million; NTPS annual burden hours 46,749 biennial or 23,374 annual hours costing $1.1million plus $7 million administrative costs. Total annual costs equal $56.3 million ($6.3 + $41.9 + $8.1 = $56.3 million).

5 The data for EDFacts are described at: “SY 2017-18 Non-XML File Specifications,” 2018, available at www2.ed.gov/about/inits/ed/edfacts/sy-17-18-nonxml.html.

6 “All public schools” includes charter schools.

7 NAEP is a biennial assessment so that NAEP can be used every two years to place state assessment scores on a common scale.

8 The CCD data collection is described at Common Core of Data (CCD; 2019).

9 The Elementary and Secondary Information System that NCES provides online to analyze the CCD data is available at https://nces.ed.gov/ccd/elsi/.

10 The quotation about the importance of the CCD data tool is from https://nces.ed.gov/ccd/.

11 The content of the Civil Rights Data Collection (CRDC) is described at “CRDC Frequently Asked Questions” (2018), available at https://ocrdata.ed.gov/downloads/FAQ.pdf.

12 The Civil Rights Data Collection (CRDC) collects school-by-school expenditure data broken out by expenditures for school personnel salaries and non-personnel expenditures. School personnel expenditures are separately shown for teachers and other personnel. The teacher and non-personnel expenditures are reported for totals and for the portion funded with state or local funds. Note also that the 2018–19 school report cards will include per-pupil expenditures for the school as a required reporting item.

13 The Civil Rights Data Collection tool to analyze the civil-rights school data online is available at https://ocrdata.ed.gov/.

14 A national study (Jordan & Miller, 2017) looking at the incidence of chronic absenteeism by the percentage of students in a school from low-income income families by grade level had to merge the CRDC data on school chronic absenteeism rates with the Common Core of Data item on the percent of students receiving free and reduced-price lunch.

15 The National Teacher and Principal Survey is described at National Teacher and Principal Survey (NTPS, 2018). Retrieved from https://nces.ed.gov/surveys/ntps/.

16 The source of the 8,300 public school number, including charters for the 2015–16 NTPS sample, is Taie and Goldring (2017). Characteristics of Public Elementary and Secondary School Teachers in the United States: Results from the 2015–16 National Teacher and Principal Survey. First Look. NCES 2017-072. National Center for Education Statistics.

17 The PowerStats data-analysis tool to analyze the National Teacher and Principal Survey online is available at https://nces.ed.gov/datalab/powerstats/percentdistribution.aspx.

18 “Unlike SASS, the 2015–16 NTPS is not explicitly designed to produce state-level estimates.” The source is the NCES website for the methods and procedures description of the National Teacher and Principal Survey, available at https://nces.ed.gov/surveys/ntps/methods1516.asp.

19 Because the CCD and EDFacts enrollments may be for different times in the school year, it may be helpful to retain the EDFacts enrollments to compute the percentage of students tested.

20 The NAEP Data Explorer is accessed at www.nationsreportcard.gov/ndecore/xplore/nde.

21 The CRDC does not collect state-level data. The only source of state-level estimates are those the OCR computes from the submitted school or district-universe data.

22 U.S Department of Education functional statement for the Immediate Office of the Assistant Secretary of the Office of Planning, Evaluation and Policy Development. Retrieved from www2.ed.gov/about/offices/list/om/fs_po/opepd/office.html.

23 What can be learned from international benchmarking of improved educational systems is examined in Mourshed, Chijioke, and Barber (2010). Learning from domestic benchmarking is illustrated by the five factors critical to school improvement identified by comparing Chicago public schools that most improved with those that did not in a benchmarking study by Bryk, Sebring, Allensworth, Easton, and Luppescu (2010).

24 Also, the estimated average socioeconomic status of school districts is improved over school lunch by tagging school districts with a composite socioeconomic index of six measures drawn from the American Community Survey (Reardon, 2017).

25 The Urban Institute has taken an important step to improving public access to education data in its beta Education Data Portal (Urban Institute, n.d.). However, the two current K–12 education data files accessible on this site are the currently separate CCD and CRDC files, not the combined files that are the focus of this article. There are also no data-analysis tools on the site.

26 For example, NAEP has responded to evaluation evidence that has shown some states excluding disproportionately high proportions of its limited English-proficient (LEP) populations from their NAEP sample (U.S. Department of Education, 2009). Also, it responded to findings that the numerous noncognitive variables that NAEP collects were largely an untapped national information resource that warranted greater NAEP analyses and dissemination of findings (Smith et al., 2012).

27 NCES could immediately jump-start its school-data improvement efforts by publicizing the current set of available online tools to the public. We only discovered these tools by accident in conducting extensive research of Education Department data tools. Adding data tools is a simple first step that can immediately expand and improve the usefulness of ED’s large school-data collections and its about $55 million annual investment.