On Learning and Testing of Counterfactual Fairness through Data Preprocessing

Abstract

Machine learning has become more important in real-life decision-making but people are concerned about the ethical problems it may bring when used improperly. Recent work brings the discussion of machine learning fairness into the causal framework and elaborates on the concept of Counterfactual Fairness. In this article, we develop the Fair Learning through dAta Preprocessing (FLAP) algorithm to learn counterfactually fair decisions from biased training data and formalize the conditions where different data preprocessing procedures should be used to guarantee counterfactual fairness. We also show that Counterfactual Fairness is equivalent to the conditional independence of the decisions and the sensitive attributes given the processed nonsensitive attributes, which enables us to detect discrimination in the original decision using the processed data. The performance of our algorithm is illustrated using simulated data and real-world applications. Supplementary materials for this article are available online.

KEYWORD:

• Causal inference
• Conditional independence test
• Fairness learning
• Machine learning ethics
• Structural causal model

Supplementary Materials

The supplementary materials provide proofs of Theorems 1 and 2, and additional simulation and real data analysis results.

Notes

1 We should note that the causal interpretation of demographic attributes such as gender and race has long been contested (VanderWeele and Robinson 2014; Glymour and Glymour 2014; Kasirzadeh and Smart 2021), part of the reason is that we cannot define a reasonable hypothetical intervention on those attributes. Here we adopt the same perspective as Kusner et al. (2017) and make the intervention on the perception of the demographic attributes. That is, the female student is not made into a boy biologically, but people’s perception of her is intervened (note we use the words ``treated as a boy'').

2 https://archive.ics.uci.edu/ml/machine-learning-databases/adult/

3 https://github.com/propublica/compas-analysis