Harmonizing Disparate Data across Time and Place: The Integrated Spatio-Temporal Aggregate Data Series

Abstract

In this article, the authors describe a new data infrastructure project being developed at the Minnesota Population Center. The Integrated Spatio-Temporal Aggregate Data Series (ISTADS) will make it easier for researchers to use publicly available aggregate data for the United States over a time span that covers virtually the entire life of the nation: 1790–2012. In addition to facilitating access and ease of use, ISTADS will facilitate the use of these various data sets in mapping and spatial analysis.

Keywords:

• aggregate data
• census data
• data integration
• GIS

Notes

The Integrated Spatio-Temporal Aggregate Data Series project at the Minnesota Population Center, University of Minnesota, is supported by a grant from the National Institutes of Health, 1RO1 HD057929.

1. The NHGIS includes an extraordinary body of statistical information and documentation, including approximately 3 terabytes of statistical data accompanied by 5 million lines of XML-encoded codebooks, 100 gigabytes of indices to the data, and more than 200,000 polygons describing historical census geography.

2. Boundary files for states/territories and counties are available for the entire time series of 1790 to the present. Census-tract boundary files are available from 1910 onward. The NHGIS also provides electronic high-quality boundary files for other units of census geography, such as congressional districts, zip code tabulation areas, voting districts, trust land, standard consolidated statistical areas, places, minor civil divisions, county subdivisions, blocks, and block groups. For the 1990 and 2000 censuses, specialized geography boundary files are provided for American Indian areas, Alaska native areas, and Hawaiian homelands. For metropolitan areas, boundary files are available for urbanized areas, New England county metropolitan areas, and—for the period 1950–2000, metropolitan statistical areas (called “standard metropolitan statistical areas” in pre-1950 data).

3. In the case of 1960 census and American Community Survey, conversion of the data to NHGIS format will be supported by other grants, NIHHD041575 and NSF BCS-0648005, respectively.

4. The necessary data in this instance would include an aggregate file with school attendance data by single years of age paired with a microdata file from the same census that identifies PUMAs. Although such data do not currently exist, we could potentially construct them for 1960 as part of the census recovery project.

5. The DDI is described at http://www.icpsr.umich.edu/DDI/.

6. The IPUMS project (Perlmann 2003) has already compiled most of these materials and is now converting them to marked-up structured documents that make machine-processing comparatively easy. We plan to make this part of the metadata system fully interoperable between the two projects, so we can create and maintain a single set of documents that will serve both the microdata and aggregate data series.

7. The level of detail in the TIGER/Line data (U.S. Census Bureau 2008), which is the basis of NHGIS files, is generally appropriate for maps drawn at scales from about 1:50,000 to 1:100,000. Most visualizations of NHGIS data will occur at smaller scales. For visualization at smaller scales, the level of detail of NHGIS data is too great, yielding undesirable graphical effects and unnecessarily large file sizes.