The pseudodiagnosticity trap: Should participants consider alternative hypotheses?

Abstract

The pseudodiagnosticity task has been used as an example of the tendency on the part of participants to incorrectly assess Bayesian constraints in assessing data, and as a failure to consider alternative hypotheses in a probabilistic inference task. In the task, participants are given one value, the anchor value, corresponding to P(D1|H) and may choose one other value, either P(D1|¬!H), P(D2|H), or P(D2|not;!H). Most participants select P(D2|H), or P(D2|¬!H) which have been considered inappropriate (and called pseudodiagnostic) because only P(D1|¬!H) allows use of Bayes' theorem. We present a new analysis based on probability intervals and show that selection of either P(D2|H), or P(D2|¬!H) is in fact pseudodiagnostic, whereas choice of P(D1|¬!H) is diagnostic. Our analysis shows that choice of the pseudodiagnostic values actually increases uncertainty regarding the posterior probability of H, supporting the original interpretation of the experimental findings on the pseudodiagnosticity task. The argument illuminates the general proposition that evolutionarily adaptive heuristics for Bayesian inference can be misled in some task situations.

Keywords:

• Bayes' theorem
• Diagnostic reasoning
• Hypothesis testing
• Pseudodiagnosticity
• Probabilistic inference.

Notes

We are grateful to David Over and several referees for comments on earlier drafts. We thank Vincenzo Crupi for an advance copy of their paper, and for extensive discussions of the issues at the Sixth International Conference on Thinking. The third author was supported by the Austrian Science Foundation within the LogICCC programme of the European Science Foundation.