How Large Are Your G-Values? Try Gosset’s Guinnessometrics When a Little “p” Is Not Enough

Abstract

A crisis of validity has emerged from three related crises of science, that is, the crises of statistical significance and complete randomization, of replication, and of reproducibility. Guinnessometrics takes commonplace assumptions and methods of statistical science and stands them on their head, from little p-values to unstructured Big Data. Guinnessometrics focuses instead on the substantive significance which emerges from a small series of independent and economical yet balanced and repeated experiments. Originally developed and market-tested by William S. Gosset aka “Student” in his job as Head Experimental Brewer at the Guinness Brewery in Dublin, Gosset’s economic and common sense approach to statistical inference and scientific method has been unwisely neglected. In many areas of science and life, the 10 principles of Guinnessometrics or G-values outlined here can help. Other things equal, the larger the G-values, the better the science and judgment. By now a colleague, neighbor, or YouTube junkie has probably shown you one of those wacky psychology experiments in a video involving a gorilla, and testing the limits of human cognition. In one video, a person wearing a gorilla suit suddenly appears on the scene among humans, who are themselves engaged in some ordinary, mundane activity such as passing a basketball. The funny thing is, prankster researchers have discovered, when observers are asked to think about the mundane activity (such as by counting the number of observed passes of a basketball), the unexpected gorilla is frequently unseen (for discussion see Kahneman 2011). The gorilla is invisible. People don’t see it.

Keywords:

• Balanced design
• Economic approach to uncertainty
• External validity
• Fisher’s p
• Randomization
• Reproducibility
• Statistical significance
• “Student’s” t

Acknowledgment

Sincere thanks to the editors and reviewers, together with Roosevelt colleagues Gary Langer and Justin Shea, for commenting on a previous version of this paper. At the Guinness Storehouse Museum (Diageo) I have enjoyed for many years the benefit of assistance from archivist Eibhlin Colgan. The seeds of the present paper were sown during a 2017 sabbatical visit at Trinity College Dublin (TRISS Research Institute) and the University of Oxford, New College. Many thanks to Ronan Lyons (Trinity College) and to Warden Miles Young (New College) for making those visits possible. Any errors are my own.

Notes

1 Harold Hotelling (1930, p. 189), a vice president of the American Statistical Association and a teacher of many leading economists and statisticians, wrote: “I have heard guesses in this country, identifying ‘Student’ with Egon S. Pearson and the Prince of Wales.”

2 See also Student 1942; Pearson 1990; Ziliak 2014, 2010a, 2010b; Ziliak and Teather-Posadas 2016.

3 https://plato.stanford.edu/entries/probability-interpret/

4 Deming (1978, p. 879). Deming said he learned the technique from Neyman (1934). In the seminal article Neyman demonstrates the statistical and economic advantages of stratified sampling over random sampling (Neyman 1934, pp. 579-585). Neyman credits the idea of “purposive selection” to earlier writers, such as Bowley and Gini and Galvani.

5 Deming (1978, p. 880-881), Tippett (1958, p. 356). In a Riesling vine-and-wine experiment, Meyers, Sacks, van Es, and Vanden Heuvel (2011) used blocking, balancing, and repetition (at n = 3 vineyards) to reduce sample size requirements by up to 60%.

6 Lavine and Schervich (1999) caution that Bayes factors can sometimes lead to incoherence in the technical statistical sense of that term.