Towards an empirically informed normative Bayesian scheme-based account of argument from expert opinion

Abstract

This article seeks, first, to show that much of the existing normative work on argument from expert opinion (AEO) is problematic for failing to be properly informed by empirical findings on expert performance. Second, it seeks to show how, with the analytic tool of Bayesian reasoning, the problem diagnosed can be remedied to circumvent some of the problems facing the scheme-based treatment of AEOs. To establish the first contention, we will illustrate how empirical studies on factors conditioning expert reliability can be drawn upon to re-construct. Walton’s critical questions matching the scheme of AEOs. To establish the second contention, we will illustrate how Walton’s re-constructed set of critical questions can be formalized within a Bayesian network. Finally, we will highlight how the specific ways in which the Bayesian framework we propose is both continuous with and distinct from the models of source reliability put forward by theorists like Bovens and Hartmann (2003).

Keywords:

• Arguments from expert opinion
• Bayesian networks
• task characteristics
• expertise
• informal argumentation
• source reliability

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1 ‘Mostly’ because some of those works have made passing remarks that are critical of the outstanding CQs. For instance, Hahn and Hornikx (2016) suggest that the set of CQs matching the scheme of ‘appeal to popular opinion’ should incorporate a question examining the extent to which verdicts made by group members are independent of each other.

2 While Walton apparently fails to appreciate the importance of systematically integrating his list of CQs with the findings of empirical research on expert performance, there is evidence that he is not completely oblivious to the relevance of such research (e.g., Walton, 199780, p.151). In addition, that for the most part argumentation theorists have not been successful in coming up with a principled account for the construction of CQs has been recognized by scholars working in the field for some time. Hahn and Hornikx, (2016) suggest that this can be achieved through combining the scheme-based approach with Bayesian argumentation. Yu and Zenker (2020) recently developed a set of meta-criteria that purportedly will provide theorists with the requisite guidance in constructing a complete list of CQs for each of the specific argument schemes.

3 One should not be misled by this example that all long-term predictions are doomed to fail or bound to be less reliable than short-term ones. Long-term events do not necessarily belong to environments of zero/low validity. Some long-term phenomena are highly regular and thus can be predicted precisely using formulae—the return of Halley’s Comet is a case in point.

4 Walton aside, while argumentation theorists have traditionally paid attention only to the matching of field (e.g., Fearnside & Holther, 1959; Crossley & Wilson, 1979; Russow & Curd, 1989), there are some notable exceptions. For instance, Hinton (2018, p. 214) holds that what matters is whether the expert has genuine expertise in the specific task at hand.

5 We used the software AgenaRisk (2018) to generate the figures in this paper.

6 The exposition of BNs here is heavily drawn from Fenton and Neil (2013, p. 141).

7 Throughout this paper, in assigning CPTs for the root nodes, we will assume the PoI. We acknowledge that this principle is controversial and is rejected by some Bayesians on the grounds that, inter alia, it is highly sensitive to how a proposition is individuated and so easily leads to contradictions. As our main purpose is to illustrate how the present BN works, we are not going to engage with the debate surrounding the plausibility of the PoI.

8 Probabilities can be understood as degrees of beliefs (subjective probabilities or credence) or as objective chances (that is, frequencies and correlations in the world independent of what people believe). In explaining our BN, we are interpreting all probabilities as degrees of beliefs. There are various ways of explaining the connection between objective chances and subjective probabilities. On one influential view that is sometimes called the ‘Principal Principle’, when objective chances are known, an agent’s subjective probabilities should match these chances (Lewis, 1980).

9 In this paper, we will focus only on expert testimonies. In formal epistemology, there are other models of source reliability developed for other applications such as explicating actual scientific practice. For the debate about what formal models can best account for the reliability and errors of scientific instruments, see for example Claveau (2013) and Osimani and Landes, (2021).

10 In one aspect, Harris et al.’s model is more fine-grained than ours. For simplicity, we put aside the trustworthiness question and hence do not model it in our BN. Our BN, however, can be easily extended to take this into consideration. We may, for example, add a root node (Trustworthiness (T)) and draw an arc from it to the E asserts that H node, with a CPT showing how E’s trustworthiness may probabilistically influence the credibility of E’s judgment. Indeed, the original model can be thought of as being a special case of this extended one where T has not been in doubt (i.e., E’s trustworthiness is always ‘True’). Thus, the NPT for the node E asserts H of the extended model is identical to the NPT in the original one when T is true.