The latent distribution of a rating observed

ABSTRACT

While extensive literature shows that the rating assigned by a critic or judge to a wine is one draw from a latent distribution, little has been published about the shape of that distribution. This article presents a derivation and test of a discrete and bounded probability mass function (PMF) that describes the distribution of the rating that a judge assigns to a wine. That PMF and the ratings that 72 wine judges assigned to blind triplicates in a commercial wine competition show that judges’ ratings are not identically distributed and that variance in ratings is a function of both the wine and the judge. Some wines are more difficult to rate consistently than others. Seventy percentage of judges reduce the variance due the wine alone, the standard deviation of the rating that a judge assigns averages 1.3 out of 10 ratings, and that deviation is significantly less than the standard deviation of random draws. The PMF and those results can be employed to improve wine-related and perhaps other taste-related research by considering the latent distribution that surrounds a rating observed.

KEYWORDS:

• Wine
• judge
• ratings
• statistics
• random

Acknowledgements

The author thanks an anonymous reviewer for insightful and constructive comments and thanks Robert Hodgson for the California State Fair Commercial Wine Competition data.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 A similar but much more complex analysis could be made using a PMF for a set of ranks or ordinal ratings.

2 A truncated normal distribution is bounded but continuous. See for example Olive (2005, p. 109). In addition to being continuous, it can also be impractical in this application because it allows ${\sigma }^2$ to be negative and thus $\sigma$ to be an imaginary number.

3 Even with analytical data on wines and in some cases judges’ qualifications, none of Baker and Amerine (1953), Frost and Nobel (2002), Cortez et al. (2009), Thompson and Mutkowski (2011), Nachev and Hogan (2013), Lemionet et al. (2015), and Xu and Wang (2017) were able to explain more than 40% to 60% of the variance, or to obtain statistically significant predictions, of the ratings that judges assign. Huang (2018) obtained better results, but only for assignment into one of two categories. Malfeito-Ferreira et al. (2019) did find significant correlation between ratings and some of judges’ descriptors. See Cortez et al. (2009) data, also used by Lemionet et al. (2015) and Huang (2018), at UC Irvine (2020) for analytical data (fixed acidity, volatile acidity, citric acid, residual sugar, chlorides, free sulfur dioxide, total sulfur dioxide, density, pH, sulphates and alcohol) on 4898 wines along with a rating of 0–10 for each wine.

4 The MATLAB code for a quadratic-search MLE written by the author, and the results of several tests using hypothetical data, are available on request.

5 The implications of Equation (1C) are consistent with Shannon (1948)’s notion of information entropy and that a signal processor can either increase or decrease that entropy. See also Rioul (2008, pp. 45–48). Shannon’s discrete definition of entropy, $H=-\sum _{min}^{max}p\left(x\right)\ln \left(p\left(x\right)\right)$, does not yield a closed-form expression for $\hat{\sigma }$ but H and $\hat{\sigma }$ are related. Both are zero when noise is zero (one outcome is certain) and both are a maximum when noise smothers a signal (every outcome has the same probability, a uniform distribution).

6 The author took the UC Davis class and test in 2018. The test involved blind tasting and assessment of various wines and testing of an ability to match a few wines when re-poured in a different order. The random probability of being able to match the order of three wines is 1/3! = 0.17. That matching test is not standardized. The more different the wines, the easier the match. The author also holds a WSET Level 3 certification.

7 Judge #1 in Panel #9 was culled because every rating for every wine reported for that judge was Bronze-.

8 For the log likelihood of the MLE result ($\mathcal{L}$) and the null hypothesis (${\mathcal{L}}_o$), the likelihood ratio statistic (LRS) is $LRS=-2\left({\mathcal{L}}_o-\mathcal{L}\right)$. For the chi-square distribution of the LRS, the degrees of freedom are the difference in the number of parameters between the tested and null models. LRS = −2 (−1989 – (−1318)) = ,342. For 3 (72) = 216 degrees of freedom, the p-value of that LRS is less than 10⁻⁹.

9 LRS = −2 (−1318 – (−1291)) = 54. For 72 degrees of freedom, the p-value of that LRS is 0.94.

10 The mean of a discrete bounded uniform random distribution simplifies to (min + max) / 2 and the variance simplifies to $\left(\right(\mathrm{m}\mathrm{a}\mathrm{x}-\mathrm{m}\mathrm{i}\mathrm{n}+1)\hat{2}+1)/12$.

11 For example, in the extreme case of a substantially flawed wine, the author has participated in professional tastings in which wines have been found to have turned to vinegar, were very obviously flawed and were given the lowest possible rating by everyone present.