Principles and Practices for a Federal Statistical Agency: Why, What, and to What Effect

Abstract

The Committee on National Statistics at the National Academy of Sciences/National Research Council first published Principles and Practices for a Federal Statistical Agency in 1992 in response to requests from Congress and others about what constitutes an effective statistical agency. Beginning in 2001, the committee has regularly updated P&P every 4 years so that a new edition is available for incoming political officials. The document helps protect the independence and objectivity of federal statistics in the highly decentralized U.S. statistical system. This article begins with a brief history of the federal statistical system and the difficulties that its decentralized structure presents for innovation and coordination, followed by an overview of the committee. The article then discusses the why and what of the first through fifth editions of P&P, highlighting major changes in each edition to respond to changes in society, the legal environment, and statistical methods and data collection techniques. The article concludes that P&P has been of value to the federal statistical agencies, to the Office of Management and Budget, and in congressional oversight.

KEY WORDS::

• Confidentiality protection
• Data dissemination
• Independence
• Multiple data sources
• Privacy
• Statistical policy

The Committee on National Statistics (CNSTAT), established at the National Academy of Sciences/National Research Council (NAS/NRC) in 1972, has had a bird's-eye view of the statistical apparatus of the U.S. government over the past 40 years. During that time, CNSTAT has sought to further the ability of a highly decentralized federal statistical system to provide relevant, timely, accurate, and cost-effective information for public policy and public understanding and to bolster the system's well-deserved reputation for objectivity and nonpartisanship. This article, after some background about the U.S. statistical system and the committee, discusses the rationale, history, and impact of the committee's signature white paper, Principles and Practices for a Federal Statistical Agency, or P&P (also known as “the purple book”), now in its fifth edition (National Research Council 1992, 2001, 2005, 2009a, 2009b, 2013). This article draws on Eddy, Citro, and Cork (2010); see also the CNSTAT Web site, www.nationalacademies.org/cnstat/, which describes current CNSTAT membership, staff, and projects and lists all CNSTAT publications.

THE FEDERAL STATISTICAL SYSTEM

Historical Development

Decentralization is a hallmark of the U.S. statistical system, in contrast to other countries, which tend to have more centralized structures for production of official statistics—for example, Statistics Canada. In the United States, as policy issues moved onto the federal stage, new cabinet departments were established, with statistical functions close behind. In fact, the United States pioneered official statistics on many topics, in some cases establishing a statistical office before establishing a full-fledged department. This decentralized structure has significant benefits, highlighted in the next section. It also, however, presents barriers to effective coordination of statistical programs and to ensuring that individual statistical agencies are able to remain independent from political and other undue external influence and otherwise live up to the fundamental principles and practices for a federal statistical agency set forth in P&P.

The foundations of the federal statistical system are in the U.S. Constitution, ratified in 1789. Article 1, Section 2, mandates a decennial census of population; the first such census was conducted in 1790 under the supervision of the Secretary of State with the primary purpose to provide the basis for apportioning seats in the U.S. House of Representatives. For censuses from 1850 through 1900, a Census Office was set up to manage the increasingly complex enumeration and compilation of census statistics but disbanded once the results were published. A permanent Census Bureau was established in 1902 (see Anderson 1988). Article 1, Section 9, stipulates that “a regular Statement and Account of the Receipts and Expenditures of all public Money shall be published from time to time.” From this authority flowed the early development of statistics on foreign trade, needed because the federal government relied heavily on tariffs for revenues in the 19th century, and put on a regular footing in 1820 in the Treasury Department.

Other important early developments in federal statistics involved agricultural and education statistics. A Census of Agriculture was first conducted in 1840, and the U.S. Department of Agriculture, called “the people's department” by President Abraham Lincoln, was established in 1862, reflecting the continued importance of agriculture to the nation's economy and the development of large mechanized farm operations with a growing need for crop forecasts and other statistics. The new department set up a Division of Statistics in 1863. An Office of Education Statistics was established in 1867 as part of a new Department of Education. The office continued even when the department was abolished 2 years later, to meet the need for education statistics that flowed from such initiatives as the federal investment in land-grant colleges during the Civil War.

Today, the federal statistical system consists of a small coordinating office in the U.S. Office of Management and Budget (OMB), 13 major statistical agencies with missions to provide a broad range of publicly available statistics in specific topic areas, and over 100 other agencies of the U.S. government that carry out statistical activities amounting to $500,000 or more of direct funding per year, including not only survey and census design and data collection, but also data analysis, forecasting, and modeling. The total statistical budget for all of these agencies was over $6 billion in fiscal 2014, amounting to about 0.04% of GDP (about twice that in decennial census years) and about $20 per U.S. resident. The major statistical agencies accounted for about 37% of the total (U.S. Office of Management and Budget 2013).

The 13 major statistical agencies and their departments are:

• Bureau of Economic Analysis (BEA), U.S. Department of Commerce.

• Bureau of Justice Statistics (BJS), U.S. Department of Justice.

• Bureau of Labor Statistics (BLS), U.S. Department of Labor.

• Bureau of Transportation Statistics (BTS), U.S. Department of Transportation.

• Economic Research Service (ERS), U.S. Department of Agriculture.

• Energy Information Administration (EIA), U.S. Department of Energy.

• National Agricultural Statistics Service (NASS), U.S. Department of Agriculture.

• National Center for Education Statistics (NCES), U.S. Department of Education.

• National Center for Health Statistics (NCHS), U.S. Department of Health and Human Services.

• National Center for Science and Engineering Statistics (NCSES), National Science Foundation.

• Office for Research, Evaluation, and Statistics (ORES), U.S. Social Security Administration.

• Statistics of Income Division (SOI), U.S. Treasury Department.

• U.S. Census Bureau, U.S. Department of Commerce.

Benefits of the Federal Statistical System

The benefits of the statistics produced by the U.S. statistical system for data users in all sectors are manifold. Just a few examples of the rich array of policy-relevant information from statistical agencies are as follows:

• The American Housing Survey (formerly the Annual Housing Survey, AHS) provides extensive information on the nation's housing stock for a national sample and rotating samples of metropolitan areas, including characteristics of the housing unit, household, and neighborhood. The AHS began in 1973; it became biannual in 1985.

• The Current Population Survey (CPS) provides closely watched information on monthly employment and unemployment. The CPS Annual Social and Economic Supplement provides yearly information on median household income, poverty rates, and health insurance coverage, and other supplements to the CPS provide valuable information on food insecurity, volunteering, and other topics. The CPS dates back to the Sample Survey of Unemployment, conducted monthly by the Work Projects Administration from 1940 to 1942 (Duncan and Shelton 1978, pp. 47, 54–55).

• The National Crime Victimization Survey since 1972 has provided comprehensive information on crimes against people and property, including those not reported to the police.

• The National Health and Nutrition Examination Survey provides a rich array of health information based not only on survey responses, but also on measurements conducted in mobile trailers equipped for comprehensive physical examinations.

• The National Income and Product Accounts since 1934 has provided integrated information on gross domestic product and income for the nation, regions, industries, and products.

In the process of producing a vast cornucopia of information, statistical agencies have pioneered advances in concepts, methods, and technology for data collection, processing, estimation, and dissemination. Just a few examples are as follows:

• Herman Hollerith (later involved in founding IBM) invented punch cards for use in processing the 1890 census by mechanical means.

• Probability sampling was introduced to reduce burden on respondents and improve data quality for federal statistics beginning in the 1940 population and housing census and the 1940–1942 predecessor to the CPS.

• The Census Bureau acquired the first commercially made computer, UNIVAC1, in time to assist in processing the 1950 census—UNIVAC1 is now in the Smithsonian.

• NCES pioneered longitudinal surveys beginning with the High School Class of 72—such surveys provide rich data for multivariate analysis by following individuals over time.

• The Census Bureau moved the collection of socioeconomic characteristics for small geographic areas from the decennial census long-form sample to continuous measurement via the American Community Survey, which became operational in 2005.

Coordinating Federal Statistics

Alongside the evolution of the federal statistical system, proposals to centralize it have been put forward by commissions, congressional committees, and executive branch working groups (Norwood 1985, chap. 2). While recommendations for centralization have never been adopted, there are instances of successful initiatives—initially catalyzed by war and depression—to strengthen coordination of the system. In 1933, an interagency Central Statistical Board was established with considerable review powers over the government's statistical activities. The Board was moved into the Budget Bureau in 1939. That Bureau was itself established in 1921, becoming the OMB in 1970. The Federal Reports Act of 1942 codified the authority for the Budget Bureau to coordinate and oversee the work of federal statistical agencies and provided that no federal agency could collect data from 10 or more respondents without approval by the budget director.

In 1977, the OMB statistical policy functions were split in two, with statistical standards and policy transferred to the Commerce Department and data collection forms review remaining in OMB. The 1980 Paperwork Reduction Act established a new Office of Information and Regulatory Affairs (OIRA) in OMB, with responsibility not only for statistical forms’ clearance, but also for statistical policy moved back from Commerce. In 1995, the reauthorization of the Paperwork Reduction Act established the Interagency Council on Statistical Policy (ICSP), chaired by the chief statistician, who heads the Statistical and Science Policy (SSP) Office in OIRA. Today the ICSP consists of the chief statistician, the 13 major statistical agencies identified above, and the Office of Environmental Information in the Energy Information Administration.

The SSP works hard and energetically with the ICSP and other federal agencies to develop and update statistical standards, such as standards for release of statistical data and definitions of industries, occupations, metropolitan statistical areas, and race and ethnicity, to establish statistical policy in such areas as data confidentiality and survey protocols, and to coordinate the work of the vast U.S. statistical apparatus. A major achievement was the passage of the Confidential Information Protection and Statistical Efficiency Act (CIPSEA), Title V of the E-Government Act of 2002, which codified confidentiality protection for data collected for statistical purposes across the federal government and provided for some kinds of data sharing among BEA, BLS, and the Census Bureau. In these and other efforts, however, the SSP is handicapped by having a small staff of only half a dozen professionals augmented by staff from other agencies who generally serve short details. The size of the staff is much smaller than the Central Statistical Board during its heyday.

Structure Impedes Statistical Innovation and Coordination

As valuable as federal statistical agencies are to data users, there are undoubtedly opportunities to improve the relevance, timeliness, accuracy, and cost-effectiveness of federal statistics by such initiatives as linking surveys in different agencies in various ways to enhance the richness of the data, reducing duplication among surveys in different agencies, and proactively sharing state-of-the-art methods for various aspects of data collection, processing, estimation, and dissemination, and other means. While the statistical agencies do seek out such opportunities, there are structural factors stemming from decentralization, which impede their efforts.

Today many of the major statistical agencies are buried in their departments. Only BLS and EIA report to their cabinet secretary, while other agencies are two and three levels down in the organization chart. This layering can not only affect the ability of agencies to make an effective case for their priorities, including ways to collaborate with other agencies, but also make it more difficult in some instances to maintain their independence from political influence. Moreover, agencies in the different departments are subject to department-specific constraints on computer system and website design standards, legal requirements for contracting and staffing, and other aspects of their work, all of which impede cross-agency coordination.

Congressional oversight for the statistical agencies is fragmented. For example, BEA, BJS, the Census Bureau, and NCSES fall under one appropriations subcommittee for review and approval of their budgets, while BLS, NCES, and NCHS fall under another. The budgets of most statistical agencies are small. The Census Bureau is the largest agency, followed by BLS, each with appropriations of greater than $500 million per year (much larger for the Census Bureau during years of heavy spending for the decennial census). At the other extreme, the budgets for BTS, NCSES, ORES, and SOI are under $50 million per year. There are legal impediments to synchronization and sharing of data among statistical agencies, even after passage of CIPSEA. A key barrier in this regard is the lack of authority for the Census Bureau to share business data that originate from tax returns with BEA or BLS. As noted above, the statistical coordinating function in OMB is limited in its available resources.

CNSTAT’S ORIGINS, MISSION, AND DOMAIN

One small entity that strives to foster collaboration among statistical agencies and improvements in methods and data is CNSTAT, which celebrated its 40th year in 2012. CNSTAT is a standing board within the Division of Behavioral and Social Sciences and Education (DBASSE) of the NRC, the operational and staff arm of the NAS. (The NAS was established as an honorific self-perpetuating nonprofit organization in 1863, with a congressional charter requiring it to respond to requests for scientific and technical advice from government agencies; the NRC was established as the NAS operating arm by a 1916 presidential executive order.) Like all NRC boards and standing committees, of which there are about 50 organized into six operating divisions, the volunteers that make up CNSTAT's membership serve as a board of directors for studies and other activities that fall within its domain.

CNSTAT establishes ad hoc study panels, convenes workshops, and undertakes other activities, such as seminars and white papers. It may also develop studies on its own initiative—particularly, cross-cutting studies of value to more than one sponsor. During the life of a study project, the committee suggests members for the study panel, receives regular updates on progress, contributes to the review of the panel or workshop report, and helps disseminate study findings and recommendations. Operating as a committee of the whole, CNSTAT regularly updates P&P. Over its 40 year history, CNSTAT has produced over 240 consensus, interim, and workshop reports.

Mission and Domain

The President's Commission on Federal Statistics, established by President Nixon and chaired by W. Allen Wallis, in its 1971 report recommended that CNSTAT be established at the NRC. The committee's original charter, as envisioned by the commission, was to “provide an outside review of federal statistical activities” and to serve as an advisory committee to the U.S. OMB “on issues of evaluation, and suggest mechanisms to promote the evaluation process” (recommendations 5–4 and 5–12). The idea was that CNSTAT would work closely with the OMB statistical policy office, which then had many more staff and could take a stronger role in coordinating statistical agency activities and in strategic planning for the system than the current chief statistician's office. OMB, with input from statistical agencies, would develop an agenda for the committee to carry out. As it developed, CNSTAT has worked cooperatively with the chief statistician in most administrations (the Reagan Administration declined to appoint a chief statistician for several years in the early 1980s), but the relationship has been one of collegiality rather than of agenda setting by OMB. CNSTAT's agenda over the years has been set by a combination of agency, congressional, OMB, and self initiatives.

CNSTAT was established with the explicit intent of advising federal statistical agencies, such as the Bureau of Economic Analysis, the Bureau of Labor Statistics, and the Census Bureau, and it has done so throughout its history. From the beginning, CNSTAT has also addressed the needs of research and policy analysis agencies, such as the Employment and Training Administration, the Food and Nutrition Service, and the National Institute on Aging. And it has considered issues of economic and social measurement in addition to data collection. Early in Miron Straf's tenure as director (1987–1999), CNSTAT adopted a formal mission statement, which is broader than its original charter from the Wallis Commission:

The Committee on National Statistics was established in 1972 to improve the statistical methods and information on which public policy decisions are based. The committee carries out studies, workshops, and other activities to foster better measures and fuller understanding of the economy, the environment, public health, crime, education, immigration, poverty, welfare, and other public policy issues. It also evaluates ongoing statistical programs and tracks the statistical policy and coordinating activities of the federal government, serving a unique role at the intersection of statistics and public policy.

This mission statement has served the committee well and has enabled it to take on projects in a wide range of subject areas and for a wide range of sponsors.

Major Themes of CNSTAT Work

Most CNSTAT projects can be categorized under six themes:

• Data Needs for Social Welfare Policies. CNSTAT has conducted studies on data needs for policy and research in such areas as environmental monitoring, immigration, rural America, family assistance and poverty, disability, health, health care, and health insurance, retirement income, an aging population, welfare reform, elder abuse, racial discrimination, and food and nutrition policies.

• The Decennial Census and Major Federal Surveys. CNSTAT has had at least one census-related study in operation on a nearly continuous basis for 30 years, beginning with a review of plans for the 1980 census. The political sensitivities in the build-up to the 2000 census were such that CNSTAT was asked to observe and evaluate that census as it occurred, from the dress rehearsal in 1998 through the completion of assessments in 2003. A similar study of the 2010 census began in summer 2009. CNSTAT has, to date, undertaken six projects on the relatively new continuous American Community Survey, which has replaced the census long-form sample, and its reviews of surveys or programs of surveys—sometimes entire statistical agencies—over the years have covered the Survey of Income and Program Participation, the Agricultural Resource Management Survey, the National Children's Study, the Office of Immigration Statistics, the National Center for Education Statistics, the Bureau of Transportation Statistics, the Bureau of Justice Statistics (including the National Crime Victimization Survey), the Census Bureau's state and local government and international trade statistics programs, NSF's surveys of research and development and science and engineering personnel, BLS's occupational safety and health statistics program, NCHS's health care provider surveys, and others.

• Economic Measurement. CNSTAT has had a sizeable portfolio on projects related to the national accounts and business surveys and statistics; recent work in this area has focused on health and healthcare accounting and science, technology, and innovation indicators. CNSTAT has also carried out major studies of important economic concepts, including those that broaden consideration of societal well-being beyond market measures—studies have considered measures of poverty, food insecurity, consumer price and cost-of-living indexes, nonmarket accounting, higher education productivity, subjective (self-reported) well-being, and civic engagement.

• Developing and Applying Statistical Methods in Diverse Policy Areas. Since the late 1990s, CNSTAT has built a portfolio of panels and workshops on statistical issues in defense acquisition and testing. It has also addressed statistical issues for other federal programs, such as the treatment of missing data in clinical trials for the Food and Drug Administration. Many of CNSTAT's statistical methods’ studies are done in collaboration with other units in the Academies. In particular, CNSTAT has regularly collaborated with the Committee on Law and Justice (CLAJ)—another DBASSE unit—on studies of measuring illegal drug markets and use, development of a national reference ballistic image database, use of information for terrorism prevention, and modernizing the nation's crime statistics.

• Evaluating Policy Models and Formulas that Affect or Implement Legislation. CNSTAT has evaluated microsimulation and other policy models that play key roles in congressional decision making about who gains and who loses consequent to changes in tax, transfer, retirement income, and health insurance programs. CNSTAT has also evaluated the use of statistical data, including surveys, administrative records, and model-based combinations of data, in formulas for allocating federal funds to states and localities for educational and other assistance.

• “Good Citizen” Projects, targeted at issues affecting the federal statistical system as a whole. Many of these have focused on confidentiality and data sharing. CNSTAT's regular production of P&P is a prime example of a “good citizen” effort.

PRINCIPLES AND PRACTICES

The 1992 (First) Edition—Why?

CNSTAT conceived the idea of codifying the essential characteristics of an effective federal statistical agency in the early 1990s in response to requests from stakeholders. Congressional staff raised the question as they were formulating legislation for a Bureau of Environmental Statistics, and the Secretary of the U.S. Department of Transportation asked CNSTAT for advice on establishing a new Bureau of Transportation Statistics, called for in the Intermodal Surface Transportation Efficiency Act of 1991.

Moreover, CNSTAT had observed that statistical agencies not infrequently face situations that tax acceptable standards for professional behavior. Examples occur when policy makers, regulators, or enforcement officials seek access to data on individual respondents from a statistical agency or when policy interpretations are added to press releases announcing statistical data. Because the federal statistical system is highly decentralized, statistical agencies must operate under the policies and guidance of officials in many departments of government. Not all of these officials are knowledgeable about what is generally accepted as proper for a federal statistical agency, and issues involving judgments about conflicting objectives also arise.

At the time, the OMB statistical policy office had not yet developed some of the system-wide legislation and guidance that it was to do beginning in the late 1990s on such topics as confidentiality protection and statistical press releases. Moreover, while there was concern that OMB should by rights develop a “principles and practices” document, it did not have the wherewithal to do so. So CNSTAT went ahead to prepare a document, taking care to solicit suggestions from many involved with federal statistical agencies. A draft of the article was discussed by the heads of some federal statistical agencies at an open meeting of CNSTAT and at a meeting of the Council of Professional Associations on Federal Statistics.

1992 Edition—Format and Content

The 1992 edition set the content and format for subsequent editions. It defined and outlined reasons for establishing a “statistical agency” (see Box 1). It identified and commented on three basic principles deemed fundamental and intrinsic to the concept of a federal statistical agency and 11 important practices that are ways and means of making the basic principles operational and facilitating an agency's adherence to them (see Box 2). To illustrate the principles and practices set forth in Part I of the document, it brought together conclusions and recommendations made in many CNSTAT reports on specific agencies, programs, and topics as commentary in Part II. Note that P&P is written at a high level of abstraction. It is not meant to be a how-to guide to best practices for data collection, processing, estimation, or dissemination, or to provide a set of data quality standards, although it discusses the need for these.

The practice that the 1992 edition labeled a “strong measure of independence” was arguably the most important concept and the most difficult to define in an appropriate manner. After all, statistical agencies, like all federal agencies, must operate within the framework of congressional, OMB, and departmental oversight. Yet they also need to maintain credibility as impartial purveyors of information not subject to partisan manipulation. The 1992 edition cited several aspects of a “strong measure of independence” that could enable statistical agencies to walk this fine line (National Research Council 1992, p. 4):

• Organizational separation from department program activities.

• Professionally qualified head; appointment for a fixed term; direct access to the secretary of the agency's department.

• Broad authority over scope, content, and frequency of data.

• Primary authority for selection and promotion of professional staff.

• Authority to release statistical information without prior clearance by policy officials.

• Authority to speak on the agency's program before Cong-ress, with congressional staff, and before public bodies.

• Adherence to predetermined schedules for release of key statistics.

• Maintenance of a clear distinction between statistical releases and policy interpretations put out by others.

Box 1—First Edition of P&P

Definition and Reasons for Establishing a Federal Statistical Agency

Definition of a Federal Statistical Agency (p. 2)

A federal statistical agency is a unit of the federal government whose principal function is the compilation and analysis of data and the dissemination of information for statistical purposes.

Reasons to Establish a Statistical Agency (p. 8)

• There is a need for information extending beyond the narrow scope of individual operating units and possibly involving other departments and agencies.

• There is a need, in fact or as a matter of credibility, to establish the independence of major data series from policy or operating control.

• There is a need to protect the confidentiality of responses.

• There is an opportunity to achieve greater efficiency or higher quality through a consolidated and more highly professional activity.

Box 2—First Edition of P&P

Principles and Practices

Three principles—

  Relevance to policy issues

  Credibility among data users

  Trust among data providers and data subjects

Eleven practices—

  A clearly defined and well-accepted mission

  A strong measure of independence

  Fair treatment of data providers

  Cooperation with data users

  Openness about the data provided

  Commitment to quality and professional standards

  Wide dissemination of data

  An active research program

  Professional advancement of staff

  Caution in conducting nonstatistical activities

  Coordination with other statistical agencies

Second Edition (2001)—Why Another?

Nine years after the 1992 edition was issued, the committee decided that it would be useful to release a revised and updated version. The committee went even further by deciding that P&P should be updated every 4 years and released on a schedule so that a new edition would be available at the beginning of a presidential administration or second term to give to new cabinet officials, congressional staff, and others.

The second edition, released in early 2001, did not change the basic principles, but it did revise and expand the discussion of some of the practices that characterize an effective federal statistical agency (see Box 3). It also brought the discussion up to date with references to recent reports by the committee and others. Appendices provided a list of selected federal statistical web sites and reproduced the Fundamental Principles of Official Statistics of the Statistical Commission of the UN, adopted in 1994 with a revised preface issued in 2013 (see http://unstats.un.org/unsd/dnss/gp/fundprinciples.aspx). Driving the adoption of the UN principles was the need for guidance for newly independent countries that were formerly part of the USSR in establishing effective and credible national statistics.

Driving the revisions to P&P in the second edition was CNSTAT's recognition of the need for statistical agencies to meet the challenges for their missions posed by such technological, social, and economic changes as the widespread use of the Internet for the dissemination and, increasingly, the collection of data, the heightened concern about safeguards for confidential information, and the information requirements of a changing economy. New and revised text addressed the reasons for establishing a federal statistical agency, the necessity for and characteristics of independence of a federal statistical agency, best practices for fair treatment of data providers, the role of the Internet in the release of data, and the importance of effective coordination and cooperation among statistical agencies to ensure that policy makers and citizens received the most accurate, relevant, and timely data possible. The second edition added a practice that charged statistical agencies to continually develop more useful data, for example, by integrating data from multiple sources.

The second edition (National Research Council 2001, pp. 15–16; see also pp. 2–3) added an explicit discussion of the need and rationale for expending public funds on federal statistics:

One of the most important reasons for establishing a statistical agency is to provide information that will allow for an informed citizenry. A democracy depends on an informed electorate. A citizen has a right to information that is relevant, accurate, and timely. Timely information of high quality is also critical to policy analysts and decision makers in both the private and public sectors. … Federal statistical agencies serve the key functions of providing a broad array of information to the public and policy makers and of ensuring the necessary quality and credibility of the data.

Private-sector organizations also provide useful statistical information,... but private markets are not likely to provide all of the data that are needed for public and private sector decision making or to make data as widely available as needed for important public purposes.

Third Edition (2005)

The third edition retained the outline and content of the second edition and the wording of the 3 practices and 11 principles. (There was one minor exception: “openness about the data provided” became “openness about sources and limitations of the data provided.”) Changes and additions to the text responded to new circumstances, such as new forms of threats to data confidentiality and individual privacy. The third edition also added an appendix that documented legislation and regulations adopted since 2001 that importantly affect the operation of federal statistical agencies. These include:

• CIPSEA (Confidential Information Protection and Statistical Efficiency Act, aka Title V of E-Government Act of 2002)

• Privacy impact assessments (PIAs) (Section 208 of E-Govt. Act)

• Information Quality Act of 2000

• OMB guidance on peer review

• Performance assessment rating tool (PART)

An important addition to the list of attributes of statistical agencies that strengthen independence was the following (National Research Council 2001, p. 7): “authority to ensure that information technology systems for data processing and analysis securely maintain the integrity and confidentiality of data and reliably support timely and accurate production of key statistics.” This attribute was added at the behest of statistical agencies in cabinet departments that were attempting to centralize information technology systems without regard to the special requirements to protect statistical data and ensure that mandated deadlines for release of key statistics, such as GDP and the unemployment rate, were met. In fact, this issue became so pressing that CNSTAT sent a letter on the topic to the director of OMB in 2003 and subsequently incorporated the substance of the letter in the 2005 edition of P&P.

Fourth Edition (2009)

The fourth edition made two important changes to the lists of principles and practices. It presented and commented on four basic principles that statistical agencies must embody to carry out their mission fully. The additional principle was that of independence, which at the behest of the statistical agencies was elevated from a practice to a principle because of continued threats to appropriate agency autonomy (see concluding section). To the list of practices that are means for statistical agencies to live up to the fundamental principles, the fourth edition added the practice of strong ongoing internal and external evaluation of statistical programs. This addition derived from CNSTAT's observation in its review of major programs that sufficient evaluation was too often missing from them.

The fourth edition added an appendix on the organization of the federal statistical system, which provided brief histories of the major statistical agencies plus examples of other statistical programs. It also enlarged the third edition appendix on legislation and regulations that govern federal statistics to include:

• Paperwork Reduction Act (PRA) and associated guidance

• 2000 Information Quality Act and associated guidelines

• 1997 OMB order providing for confidentiality of statistical information

• 2002 CIPSEA and 2007 guidance

• 2002 E-Government Act, Section 208, privacy impact assessments (PIAs)

• 2002 Federal Information Security Management Act (FISMA), Title II of the E-Government Act of 2002

• 2004 OMB peer review guidance

• 2002–2008 OMB performance assessment rating tool (PART)

• OMB statistical policy directives (metropolitan areas, race/ethnicity, industry/occupation codes, survey standards, etc.)

Box 4—Fourth Edition

Principles and Practices*

Four principles—

  Relevance to policy issues

  Credibility among data users

  Trust among data providers

  A strong position of independence

Eleven practices—

  A clearly defined and well-accepted mission

  Continual development of more useful data

  Openness about sources and limitations of the data provided

  Wide dissemination of data

  Cooperation with data users

  Fair treatment of data providers

  Commitment to quality and professional standards of practice

  An active research program

  Professional advancement of staff

  A strong internal and external evaluation program

  Coordination and cooperation with other statistical agencies

*Additions to third edition text shown in bold.

Fifth Edition (2013)

The fifth edition of P&P made several important changes in the lists of principles and practices (see Box 5). First, it responded in two ways to a continued need to clarify and strengthen the principle of independence by: (1) revising the wording of the principle of “a strong position of independence” to “independence from political and other undue external influence,” including a footnote (National Research Council 2013, p. 14) to explain that “a statistical agency actively works to obtain a broad range of external input to develop its programs; ‘undue external influences’ are those that seek to undermine an agency's impartiality and professional judgment”; and (2) adding a practice of “necessary authority to protect independence.”

The fifth edition also responded to the growing concern in the society at large about invasions of personal privacy from the public and private sectors by separating the practice of “fair treatment of data providers” into two practices: “respect for the privacy and autonomy of data providers; and “protection of the confidentiality of data providers’ information.” Finally, the fifth edition changed the practice of “coordination and cooperation with other statistical agencies” to “coordination and collaboration with other statistical agencies” to underscore the need for proactive partnerships among the agencies given increasingly constrained budgets and at the same time increasing demands for more timely and relevant statistical information.

The fifth edition expanded the discussion of most of the principles and practices in the Part II Commentary to respond to agency and reviewer suggestions. The biggest expansion occurred for the practice of “continual development of more useful data,” for which the discussion covered in some detail the current and potential future roles in statistical programs of surveys, administrative records, and selected Internet and other nontraditional data sources. The discussion noted the need for statistical agencies to move from a model of continuing long-standing survey or administrative records programs for their own sakes, to one of producing the best estimates possible in terms of relevance, accuracy, timeliness, and cost-effectiveness, which would likely require the use of multiple data sources and techniques for combining data.

Other changes in the fifth edition of P&P included:

• A dedication to Margaret Martin, first director of CNSTAT and a former president of the American Statistical Association, who helped edit the first-fourth editions; she died at age 100 in May 2013 shortly before the release of the fifth edition.

• The addition of an executive summary, available separately as “Highlights” (available at http://sites.nationalacadem-ies.org/DBASSE/CNSTAT/Principles_and_Practices_for_a_Federal_Statistical_Agency/index.htm).

• The addition in an appendix of the European Statistics Code of Practice for the National and Community Statistical Authorities (adopted 2011) (available at http://epp.eur-ostat.ec.europa.eu/portal/page/portal/product_details/pub-lication?p_product_code=KS-32–11–955).

• The addition of new entries to the appendix on relevant legislation and regulations:

  • 2009–2010 Guidance on Scientific Integrity issued by the Office of Science and Technology Policy.

  • 2010 Government Performance and Results Modernization Act (GIPRA).

TO WHAT EFFECT?

The preparation of each edition of P&P takes resources, including not only the time of CNSTAT members and staff, but also the time of agency leadership and anonymous reviewers to provide input and of editorial and production staff. Is there any way to measure the return on those resources? One indicator that the federal statistical agencies find the document of use is that, in addition to complimentary copies that CNSTAT provides of each edition as it is released, many statistical agencies routinely order 100 or several hundreds of copies to distribute to department and agency staff. P&P is used in a required class on the federal statistical system for graduate students in the University of Maryland–University of Michigan Joint Program in Survey Methodology. Heads of statistical agencies who are presidential appointees subject to Senate confirmation often cite P&P in confirmation hearings. Career staff of the OMB SSP Office use P&P in educating new political appointees in OMB and cabinet departments. Indeed, as it turned out, the lodgment of P&P at the NAS/NRC has been beneficial in that the document is not tied to any one branch of government nor to any administration or political party.

There are known instances of the use of P&P in consequential ways. The first (1992) edition was cited in a congressionally mandated study by the U.S. General Accounting Office (1995) to evaluate the performance of major statistical agencies and in a review of the federal statistical system by a former commissioner of the Bureau of Labor Statistics (Norwood 1995). Its principles informed the establishment and later assessment of a new statistical agency, the Bureau of Transportation Statistics (see National Research Council 1997).

The third edition of P&P (2005) played a role in strengthening independence of statistical agencies. Regrettably, the weight of evidence is that executive branch departments, in their efforts at message control, can too easily engage in behavior that undercuts the credibility of statistical agencies as sources of objective, nonpartisan information for use by all. There are a number of examples across U.S. history, one of which occurred in the early 1970s when White House staff sought to purge senior BLS leadership when they indicated that a decline in unemployment was likely overstated due to technical problems (see Goldberg and Moye 1985). In response, OMB issued Statistical Policy Directive No. 3, which specified publication schedules and release procedures for the unemployment rate and other key economic indicators. Available at http://www.whitehouse.gov/sites/default/files/omb/assets/omb/inforeg/statpolicy/dir_3_fr_09251985.pdf, this directive was first issued in the 1970s and was subsequently strengthened and reissued in 1985. The intent was to forestall any efforts by political officials to interfere with the timing or the substance of the release of key economic indicators or for any unauthorized person to gain advance access to the latest data.

More recently, political officials at the U.S. Department of Justice insisted on politically motivated changes to a press release of the Bureau of Justice Statistics reporting statistics for racial and ethnic groups on traffic stops and searches by police. The BJS director refused to alter the release and was transferred to another position (see National Research Council 2009a). The U.S. Government Accountability Office (2007) cited P&P in a review of this incident, and the U.S. Office of Management and Budget (2008) cited P&P in a new Statistical Policy Directive No. 4, issued in 2008, that was a response to what happened to BJS. Directive 4 covers all of the other data that statistical agencies release beyond the principal economic indicators and is designed to protect the agencies’ ability to release information on predetermined schedules with control over the content of accompanying press releases (see National Research Council 2009b: App. B).

The fourth and fifth editions, as noted above, further sought to strengthen the principle and practice of appropriate independence for statistical agencies. Both editions were endorsed by the Board of the American Statistical Association (see http://www.amstat.org/newsroom/pressreleases/ASAEndorses-PrinciplesandPractices.pdf), and the fourth edition was cited as a key document in a Statement of Commitment to Scientific Integrity by Principal Statistical Agencies. This statement was developed by the principal statistical agencies in response to a 2010 memorandum by the Office of Science and Technology Policy requiring all federal agencies to develop strategies to ensure scientific integrity in government decision making (see National Research Council 2013: App. A). The U.S. Government Accountability Office (2012) cited the entire P&P series in a report on the potential for agencies to make greater use of existing data, such as administrative records, and to achieve greater collaboration.

The history to date supports the value of Principles and Practices for a Federal Statistical Agency. It also supports the necessity for regular updating, not only to educate new cohorts of political officials, but also to keep up with changes in society, the legal environment, statistical methodology, and data collection technology that affect the nation's need for and ability to produce timely, relevant, and accurate statistics. CNSTAT staff have already begun to garner ideas for a sixth edition of P&P, to be issued in early 2017 to continue the series’ mission to assist statistical agencies and inform policy makers, data users, and others of the characteristics of statistical agencies that enable them to serve the public good.