Optimizing Individual and Collective Reliability: A Puzzle

ABSTRACT

Many epistemologists have argued that there is some degree of independence between individual and collective reliability. The question, then, is: To what extent are the two independent of each other? And in which contexts do they come apart? In this paper, I present a new case confirming the independence between individual and collective reliability optimization. I argue that, in voting groups, optimizing individual reliability can conflict with optimizing collective reliability. This can happen even if various conditions are held constant, such as: the evidence jurors have access to, the voting system, the number of jurors, some independence conditions between voters, and so forth. This observation matters in many active debates on, e.g., epistemic dilemmas, the wisdom of crowds, independence theses, epistemic democracy, and the division of epistemic labour.

KEYWORDS:

• Reliabilism
• epistemic justification
• group reliability
• jury theorem
• independence thesis

Acknowledgments

Thanks to Aude Bandini, Cameron Boult, Étienne Brown, François Claveau, Charles Côté-Bouchard, Samuel Dishaw, Megan Entwistle, Félix-Antoine Gélineau, Hilary Kornblith, Daniel Laurier, Éliot Litalien, Molly O’Rourke-Friel, Andrei Poama, Juliette Roussin, Rémi Tison for helpful comments on this project. I also thank referees for their feedback on different versions of this manuscript. This research was supported by the Groupe de Recherche Interuniversitaire sur la Normativité (GRIN), the Fonds de recherche du Québec – Société et culture (grant #268137), and the Social Sciences and Humanities Research Council (grant #756-2019-0133).

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Notes

1. See Goldman and Beddor (2016) for a survey of the debates on reliabilism. See, e.g., Cohen (2002) and Conee and Feldman (1998) on problems for reliabilist theories of justification.

2. See Beebe (2004), Goldman (1986, 1994, 2001), Roberts and West (2015) or Samuelson and Church (2015).

3. See Dietrich and List (2004), Estlund (1994, 1997), Estlund and Landemore (2018), Hedden (2017), Hong and Page (2012), Landemore (2012a, 2012b, 2013), List (2005), List and Goodin (2001), Page (2007, 2010), Pallavicini, Hallsson, and Kappel (2018), Sunstein (2006) or Surowiecki (2005). Jury Theorems are also relevant here. See de Condorcet (1976), and see Bachrach et al. (2012), Dietrich and Spiekermann (2013a, 2013b), Fey (2003), Kaniovski (2010), List and Goodin (2001), Romeijn and Atkinson (2011) or Stone (2015) on various generalizations or applications of Condorcet’s Jury Theorem.

4. See, e.g., Mayo-Wilson, Zollman, and Danks (2011) and Dunn (2018) on Independence theses. See also Hong and Page (2012), Landemore (2012a, 2012b), and Kitcher (1990, 1995).

5. See Goldman (2010), Schoenfield (2014), and Titelbaum and Kopec (2019) on standards (or methods) of reasoning.

6. See Dietrich and Spiekermann (2013a, 99, 2013b, 666) on Problem-Conditional Independence, or New Independence.

7. See Grofman, Owen, and Feld (1983) and Stone (2015) on reliability for groups of voters that have different competence levels. See Dietrich and Spiekermann (2013a, 99; 2013b, 666) on reliability for groups of voters that face different types of problems.

8. Formally: β₀ ≈ 0.999 and σ₀ = 0.5. We get these reliability levels by calculating:

${\beta }_0=\left(\begin{array}{c}5\\ 3\end{array}\right){0.99}^3\cdot {0.01}^2+\left(\begin{array}{c}5\\ 4\end{array}\right){0.99}^4\cdot {0.01}^1+\left(\begin{array}{c}5\\ 5\end{array}\right){0.99}^5\cdot {0.01}^0$

${\sigma }_0=\left(\begin{array}{c}5\\ 3\end{array}\right){0.5}^3\cdot {0.5}^2+\left(\begin{array}{c}5\\ 4\end{array}\right){0.5}^4\cdot {0.5}^1+\left(\begin{array}{c}5\\ 5\end{array}\right){0.5}^5\cdot {0.5}^0$

9. Formally: Given that β₀ ≈ 0.999 and σ₀ = 0.5, (0.65·β₀)+(0.35·σ₀)≈0.825.

10. Formally: β₂ ≈ 0.97 and σ₂ = 0.8. We get these numbers by calculating:

${\beta }_2=\sum _{j=1}^3\left[\sum _{i=j}^3\left(\begin{array}{c}3\\ i\end{array}\right){0.99}^i\cdot {0.01}^{\left(3-i\right)}\cdot \left(\begin{array}{c}2\\ 3-j\end{array}\right){0.1}^{\left(3-j\right)}\cdot {0.9}^{\left(j-1\right)}\right]$

${\sigma }_2=\sum _{j=1}^3\left[\sum _{i=j}^3\left(\begin{array}{c}3\\ i\end{array}\right){0.5}^i\cdot {0.5}^{\left(3-i\right)}\cdot \left(\begin{array}{c}2\\ 3-j\end{array}\right){0.9}^{\left(3-j\right)}\cdot {0.1}^{\left(j-1\right)}\right]$

11. Formally: Given that β₂ ≈ 0.97 and σ₂ = 0.8, (0.65·β₂)+(0.35·σ₂)≈0.91.

12. Formally: (0.65·β₂)+(0.35·σ₂)>(0.65·β₀)+(0.35·σ₀).

13. Screeners can also be asked to give a numerical score, or a rank, to the proposals (Lamont 2009, 28, 37).

14. In the puzzle, I focus on an extreme case where a juror reaches the correct answer less than 50% of the time. Lamont’s research tells us that some screeners were perceived as unreliable, not that they reached the right answer less than 50% of the time (see Lamont 2009, 38–9). However, the extreme case described in the Puzzle is instructive for understanding the more nuanced situations described by Lamont. Even if a screener is profoundly unreliable, we might still have a good reason to include him or her in the group.

15. This observation goes back to de Condorcet (1976).

16. See Lewis (1971). See Daoust (2021) for discussion in jury contexts.

17. See Hughes (2019) on epistemic conflicts from a third-personal point of view.

18. See footnote 4 and Palmira (2018, sec. 6) on the Independence Thesis and doxastic attitudes.

19. Dogramaci and Horowitz argue that, while there is a strong connection between rational standards and reliable processes, reliability is not a sufficient condition for epistemic rationality (Dogramaci and Horowitz 2016, 135).

20. A similar argument can be found in Greco and Hedden (2016). See Daoust (2017) for other objections.

21. This observation is also confirmed by Kitcher’s decision-theoretic argument. See Kitcher (1990).

22. Maskivker’s ‘better than random’ condition concerns binary choices (i.e. choices between two options, like ‘Guilty’ and ‘Not guilty’). I have also focused on binary choices throughout this article.

Additional information

Funding

This work was supported by the Social Sciences and Humanities Research Council of Canada [756-2019-0133].

Notes on contributors

Marc-Kevin Daoust

Marc-Kevin Daoust is a Maître d’enseignement at l’École de technologie supérieure (Montréal, Canada). He is also a regular member of the Interuniversity Research Group on Normativity (GRIN). His current research focuses on the (ir)relevance of ideals in ethics and epistemology.