On the Possibility of Testimonial Justice

ABSTRACT

Recent impossibility theorems for fair risk assessment extend to the domain of epistemic justice. We translate the relevant model, demonstrating that the problems of fair risk assessment and just credibility assessment are structurally the same. We motivate the fairness criteria involved in the theorems as also being appropriate in the setting of testimonial justice. Any account of testimonial justice that implies the fairness/justice criteria must be abandoned, on pain of triviality.

KEYWORDS:

• calibration
• credibility
• epistemic injustice
• equalised odds
• fair algorithms

Disclosure Statement

No potential conflict of interest was reported by the author.

Notes

1 As an anonymous referee indicated, a similar result is proved earlier in the context of psychometrics and fair selection [Borsboom, Romeijn, and Wicherts 2008].

2 Fricker came to agree that distributive aspects figure into certain legitimate notions of epistemic injustice [2010: 175]:

I hope it is clear we may think of the concept of epistemic injustice as an inclusive, generic notion, up for further exploration. In particular, it should be thought of as including distributive forms of epistemic injustice, such as unequal access to epistemic goods like information, or education. In this I agree with David Coady (this issue), who rightly affirms that distributive forms of epistemic injustice are, contrary to what I seem to say at the start of the book, distinctively epistemic injustices.

3 Recall that the conditional expectation $E_k\left(h\mid Y=0\right)$ is the expected value of $h$ with respect to the conditional probability $P_k\left(\cdot \mid Y=0\right)$ (similarly for $E_k\left(1-h\mid Y=1\right)$).

4 Concerning the base rates for credibility in particular, some standpoint theorists argue that ‘those who are subject to structures of domination that systematically marginalize and oppress them may, in fact, be epistemically privileged in some crucial respects’ [Wylie 2013: 26]. And, generally, there might be good reason to think that population base rates differ with respect to their credibility on certain topics. Consider the population of Alabama, partitioned into the set of climate scientists and its complement. There can be little doubt that credibility base rates differ for these groups on topics in climate science.

5 For completeness, we provide an elementary proof of Theorem 1 in the Appendix.

6 But see Seidenfeld [1985], for instance, for some critical remarks concerning calibration as a general norm for subjective probability.

7 At the extreme end of reliance on group base rates is profiling.

8 There are nuances here that we cannot fully address in this paper. Epistemic sacrifice in contest settings can itself lead to harmful effects on already disadvantaged groups. For example, as Borsboom et al. write [2008: 87],

consider again the scenario as it may occur in selection situations that involve minority groups. As explained […], there are empirical reasons to consider the possibility that accepted members from some minority groups may include a larger number of false positives, which may be partly responsible for the observed increased dropout rates among members of such groups. The presence of more false positives among minorities may create a perceived empirical basis for prejudice.

The precise distribution of the benefits and harms of testimonial injustice, it seems to us, is largely context-dependent. Our primary goal, however, is to indicate substantive constraints for theories of testimonial justice rather than to analyse this context-dependence.

9 This is the literature that triggered our thinking about the subject of this paper, but other established literatures are also relevant—e.g. work in psychometrics on fair selection [Borsboom, Romeijn, and Wicherts 2008], and work in labour economics on statistical discrimination [Fang and Moro 2011].

10 We were first introduced to the literature on fair algorithms, and the sorts of limitative results discussed here, by Tina Eliassi-Rad’s talk at the Decision & AI conference in Munich in July 2018. We learned about Jennifer Lackey’s views on testimonial justice from episode 98 of the Elucidations podcast shortly afterwards. Thanks to Lackey for sharing a draft of her paper with us. And thanks to Jean Baccelli, Patrick Klösel, Alex Reutlinger, Jan-Willem Romeijn, Joe Roussos, Georg Schollmeyer, Shanna Slank, Tom Sterkenberg, Reuben Stern, Greg Wheeler, audiences at FEW 2019 and the 2019 Summer School in Mathematical Philosophy for Female Students, and two anonymous referees for very constructive feedback and suggestions that helped to improve the paper a great deal.