Inference on Consensus Ranking of Distributions

Abstract

Instead of testing for unanimous agreement, I propose learning how broad of a consensus favors one distribution over another (of earnings, productivity, asset returns, test scores, etc.). Specifically, given a sample from each of two distributions, I propose statistical inference methods to learn about the set of utility functions for which the first distribution has higher expected utility than the second distribution. With high probability, an “inner” confidence set is contained within this true set, while an “outer” confidence set contains the true set. Such confidence sets can be formed by inverting a proposed multiple testing procedure that controls the familywise error rate. Theoretical justification comes from empirical process results, given that very large classes of utility functions are generally Donsker (subject to finite moments). The theory additionally justifies a uniform (over utility functions) confidence band of expected utility differences, as well as tests with a utility-based “restricted stochastic dominance” as either the null or alternative hypothesis. Simulated and empirical examples illustrate the methodology.

KEYWORDS:

• Confidence set
• Expected utility
• Multiple testing
• Stochastic dominance

Supplementary Materials

The supplementary appendix has additional (more technical) theoretical results, additional methodology, proofs not found in the main text, and an example algorithm to compute bootstrap critical values. Also provided is R code to replicate the simulation and empirical results.

Acknowledgments

Thanks to Tim Armstrong for the initial idea, and to the Cowles Foundation more generally for their hospitality. Thanks to Alyssa Carlson for helpful comments on multiple drafts, and to Saku Aura and other colleagues for feedback at a brown bag seminar. I am also grateful for the helpful feedback from seminars at The University of Chicago, Boston University, and UC Santa Cruz (especially from Stéphane Bonhomme, Jim Heckman, Azeem Shaikh, Alex Torgovitsky, Hiro Kaido, Iván Fernández-Val, JJ Forneron, Pierre Perron, Jessie Li, and Julián Martínez-Iriarte) and the 2021 North American Summer Meeting of the Econometric Society (especially Ruli Xiao, Jackson Bunting, Yuya Sasaki, and Takuya Ura), as well as feedback from anonymous reviewers, the associate editor, and editor Ivan Canay, all of which helped improve this work.

Disclosure Statement

I have no competing interests to declare.

Notes

1 R code: integrate(function(x) ((x(1-0.6)-1)/(1-0.6))^2*2*dt(x-1,1), 1, Inf)

2 R code: integrate(function(x) ((x(1-0.01)-1)/(1-0.01))^2*2*dt(x-1,2), 1, Inf)

3 https://kaplandm.github.io

4 https://docs.google.com/spreadsheets/d/1ctjNbPyAaPfaLy3dp82TGFmpZ6afnQkAnWcOjJJKhk4

5 https://scholar.google.com/scholar?as_ylo=2019&q=hara+utility+investment

6 The “veil of ignorance” (or “original position”) was popularized by Rawls, but the concept is found in earlier economics articles, combined with expected utility maximization: Vickrey (1945) writes (emphasis added), “to maximize the aggregate of such utility over the population is equivalent to choosing that distribution of income which such an individual would select were he asked which of various variants of the economy he would like to become a member of, assuming that once he selects a given economy with a given distribution of income he has an equal chance of landing in the shoes of each member of it” (p. 329); and Harsanyi (1953) writes, “Now, a value judgment on the distribution of income would show the required impersonality to the highest degree if the person who made this judgment…had exactly the same chance of obtaining the first position (corresponding to the highest income) or the second or the third, etc., …. [Such judgments] may still be interpreted as an expression of what sort of society one would prefer if one had an equal chance of being “put in the place of” any particular member of the society, so that the cardinal utility “maximized” in value judgments concerning social welfare and the cardinal utility maximized in choices involving risk may be regarded as being fundamentally based upon the same principle” (p. 435).

7 https://kaplandm.github.io