The paper “What Protects the Autonomy of the Federal Statistical Agencies? An Assessment of the Procedures in Place to Protect the Independence and Objectivity of Official U.S. Statistics” by Citro et al. (2023) provides a compelling argument for its assertions about the importance of autonomy for the federal statistical agencies, what that entails, and a discussion of the current threats to that independence. It concludes by offering three recommendations to both protect the existing degree of autonomy and to establish a greater degree of independence of the actions of these agencies from their parent Administrations in the future. It is hard to find anything to even quibble with, or to more fully develop. I certainly agree that autonomy is crucially important to a federal statistical agency, especially in the present time of intense political polarization, since without a strong sense that statistical agencies are behaving in a nonpartisan manner, the resulting data products will be viewed by many as being cherry-picked or otherwise biased to conform to the Administration’s point of view and not to the truth. Further, with the private sector’s increased production of competing estimates to those from the federal statistical agencies, one of the key distinguishing features of estimates from the federal statistical system over those from the private sector is that they are independent of any stakeholder position. Given this and the further arguments provided in Citro et al. (2023), it is very hard to disagree with the importance of the degree of autonomy described there.
However, as a formal discussant, I am required to push myself to try to find a few areas in which further development might be profitable. So here are some issues, not listed in any particular order, that may generate further discussion of this excellent paper.
To start, though I have been around federal statistics for around 40 years, it was still surprising to me (and evidently to some of the authors) that many of the 13 major statistical agencies do not operate under laws that protect their autonomy (or even their establishment). Such laws are especially important to have during times of stress, for example, economic, political, or security crises, when there can be pressures from the parent departments to, for example, make available some personally identifiable information to a different agency, change an existing methodology to a competing approach, or delay publication of a statistical data product (possibly until after an election), all for partisan reasons. Two related examples worth mentioning were the pressures on agencies to release data in response to 9–11, (e.g., pressure resulting from the investigations in response to those terrorist attacks to release special tabulations to security agencies at the block level as to the number of resident Arab Americans), and the pressure from the Nixon White House to change the method concerning the contribution of household rents to the Consumer Price Index. Given that many of the federal statistical agencies, to a great extent, rely on internal departmental MOUs and historical precedents to protect them from such pressures, which can at times be extreme, it seems likely that such protections will often be inadequate, especially during intense pressures during crises.
It should be mentioned that even laws may not be sufficient protection as a result of such pressures. This can be for at least two reasons. First, a suggested change in a methodology, which was due to an effort to change to a competing estimate that was consistent with a partisan point of view, might be represented instead as being due to an uncertainty as to what methodologies were consistent with current practice. Determining whether such a switch was reasonable due to this argument would not always be easy to resolve. A somewhat relevant example was the adjustment question that was an issue during the 1980, 1990, and 2000 censuses. There were many statisticians of the highest reputations on either side of this issue writing journal articles and providing testimony in many settings, including the Supreme Court. The Census Bureau director, Barbara Bryant, decided that the Census Bureau should employ adjustment in conjunction with the release of some of the 1990 Census data products, but her decision was overturned by the Commerce Department. I do not know whether partisan considerations played any role in the decision by the Commerce Department, but the fact that there was no professional consensus might have made this decision easier to represent as a technical one. Maybe a better example was the one provided above in the dispute on the methodology underlying the Consumer Price Index between the Bureau of Labor Statistics (BLS) and the Nixon White House. Janet Norwood made it clear to the White House that regarding the disagreement between them on how to handle rents in the estimation of inflation, if the decision went against her, she was prepared to resign. The decision was made to side with BLS and Janet Norwood felt that her stance played a role in this decision. This is something that top administrators for statistical agencies might have to consider as potential ultimate steps to protect the autonomy of a statistical agency in extreme situations.
Second, there is the question, when there are competing legal imperatives of, say, autonomy versus national security, whether any law supporting autonomy would be sufficient to argue against, say, release of some data to the FBI during a crisis. I don’t know how such conflicts would be resolved.
Next, I am pleased that the paper discussed the importance of transparency through documentation as being a “powerful check on any attempts to politicize or manipulate the numbers.” Documentation of data and methodology (hence computational reproducibility) is possibly the only way to determine whether an Administration has forced a statistical agency to change how a statistical product was produced. In addition to documenting the current methodology applied (often represented as a code with detailed comments), documenting the current input data used (possibly including documentation of any special treatment of ‘outliers’), and documenting the resulting current data products, one should also add the archiving of data, methods, and results over time, so that the time at which changes in methods and data were made can be determined.
Finally, I wish to comment on the second recommendation to strengthen the office of the Chief Statistician of the United States. As part of the discussion, it was mentioned that the current size of the staff at OMB for oversight of statistical agencies is just 4–6 people, when it was once as large as 75 people. The idea that 4–6 people can oversee what is going on in 13 principal statistical agencies and something like 100 other statistical units throughout the federal government makes absolutely no sense. We are talking about hundreds of separate programs each producing statistical data products which each need to be monitored in various ways, often in depth. This long-standing problem has been ignored though it has been raised by many others and is clearly needed. The problem is adding FTEs to the staff of the CSOTUS, which given the current budget climate is probably a nonstarter. As a result, I am doubtful that this recommendation will bring about the needed change. So one is forced to ask whether there is a way to do this without adding FTEs to the staff of the CSOTUS? Maybe through delegation of a staff person from each agency for a short stint at OMB to deal with issues orthogonal to that agency? There are likely a number of reasons this is not possible but it might be useful to discuss how this idea can be modified to make it feasible.
In conclusion, Citro et al. (2023) will certainly become THE reference in discussions of this critical issue until it is at least partially mitigated. Ensuring the autonomy of federal statistics is necessary to support the future utility of our federal statistical system.