Ambiguity, the certainty illusion, and the natural frequency approach to reasoning with inverse probabilities

Abstract

People have difficulty reasoning with diagnostic information in uncertain situations, especially when an understanding and calculation of inverse conditional probabilities (Bayes theorem) is required. While natural frequency representations of inference tasks improve matters, they suffer from three problems: (1) calculation errors persist with a majority of subjects; (2) the representation suffers from an illusion of certainty that ignores ambiguity; and (3) the costs of repeatedly applying the representation to deal with imprecision and ambiguity in inference are prohibitive. We describe a user friendly, interactive, graphical software tool for calculating, visualizing, and communicating accurate inferences about uncertain states when relevant diagnostic test information (sensitivity, specificity, and base rate) is both imperfect and ambiguous in its application to a specific patient. The software is free, open-source, and runs on all popular PC operating systems (Windows, Mac, Linux).

Keywords:

• ambiguity
• subjective expected utility
• certainty illusion
• inverse probabilities
• choice under uncertainty
• natural frequencies

Notes

1. Barbey and Sloman (2007), Brase (2002, 2008), Gigerenzer (2002), Gigerenzer and Hoffrage (1995).

2. http://en.wikiquote.org/wiki/John_Tukey.

3. Coherency of probabilities means satisfying the laws of probability at a formal level. Coherency of imprecise probabilities means that whatever imprecise and/or incomplete assessments of conditional or marginal probabilities are made are capable of being supported by a coherent probability distribution – even if that full distribution is not precisely assessed. See Lad (1996, chs. 2 and 3) or Walley et al. (2004) for a detailed exposition of the theory of imprecise probabilities.

4. This is because upward movements away from the origin on the vertical axis measure larger magnitudes for probabilities of the test being positive and the specificity itself is a conditional probability for the test being negative. Since the two probabilities P(T|S = 0) (shorthand for P(T = 1|S = 0)) and P(T = 0|S = 0) must sum to 1, once we plot a point at height P(T|S = 0) we can measure the size of the specificity P(T = 0|S = 0) downwards by the amount 1–P(T|S = 0) from the point (0,1) on the y-axis.

5. Lad (1996) has a clear and insightful development of the inference issues associated with upper and lower limits on component ‘pieces’ of coherent probability distributions. See also Walley et al. (2004). For recent research on ambiguity and ambiguity aversion see Klibanoff (2005), Nau (2007), Al-Najjar and Weinstein (2009), and Nehring (2009).