Lookahead Counterfactual Fairness

The integration of machine learning (ML) into high-stakes domains (e.g., lending, hiring, college admissions, healthcare) has the potential to enhance traditional human-driven processes. However, it may introduce the risk of perpetuating biases and unfair treatment of protected groups. For instance, the violence risk assessment tool SAVRY has been shown to discriminate against males and foreigners (Tolan et al., 2019); Amazon’s previous hiring system exhibited gender bias (Dastin, 2018); the accuracy of a computer-aided clinical diagnostic system varies significantly across patients from different racial groups (Daneshjou et al., 2021). Numerous fairness notions have been proposed in the literature to address unfairness issues, including unawareness fairness that prevents the explicit use of demographic attributes in the decision-making process, parity-based fairness that equalizes certain statistics (e.g., accuracy, true/false positive rate) across different groups (Hardt et al., 2016b; Khalili et al., 2023; 2021b; 2021a; Abroshan et al., 2024), preference-based fairness that ensures a group of individuals, as a whole, regard the results or consequences they receive from the ML system more favorably than those received by another group (Zafar et al., 2017; Do et al., 2022). Unlike these notions that overlook the underlying causal structures among different variables Kusner et al. (2017) introduced the concept of counterfactual fairness (CF), which requires that an individual should receive a consistent treatment distribution in a counterfactual world where their sensitive attributes differs. Since then many approaches have been developed to train ML models that satisfy CF (Chiappa, 2019; Zuo et al., 2022; Wu et al., 2019; Xu et al., 2019; Ma et al., 2023; Zuo et al., 2023; Abroshan et al., 2022).

However, CF is primarily studied in static settings without considering the downstream impacts ML decisions may have on individuals. Because humans in practice often adapt their behaviors in response to the ML system, their future status may be significantly impacted by ML decisions (Miller et al., 2020; Shavit et al., 2020; Hardt et al., 2016a). For example, individuals receiving approvals in loan applications may have more resources and be better equipped to improve their future creditworthiness (Zhang et al., 2020). Content recommended in digital platforms can steer consumer behavior and reshape their preferences (Dean & Morgenstern, 2022; Carroll et al., 2022). As a result, a model that satisfies CF in a static setting without accounting for such downstream effects may lead to unexpected adverse outcomes.

Although the downstream impacts of fair ML have also been studied in prior works (Henzinger et al., 2023a; Xie & Zhang, 2024a; Ge et al., 2021; Henzinger et al., 2023b; Liu et al., 2018; Zhang et al., 2020; Xie et al., 2024), the impact of counterfactually fair decisions remain relatively unexplored. The most related work to this study is (Hu & Zhang, 2022), which considers sequential interactions between individuals and an ML system over time and their goal is to ensure ML decisions satisfy path-specific counterfactual fairness constraint throughout the sequential interactions. However, Hu & Zhang (2022) still focuses on the fairness of ML decisions but not the fairness of the individual’s actual status. Indeed, it has been well-evidenced that ML decisions satisfying certain fairness constraints during model deployment may reshape the population and unintentionally exacerbate the group disparity (Liu et al., 2018; Zhang et al., 2019; 2020). A prime example is Liu et al. (2018), which studied the lending problem and showed that the lending decisions satisfying statistical parity or equal opportunity fairness (Hardt et al., 2016b) may actually cause harm to disadvantaged groups by lowering their future credit scores, resulting in amplified group disparity. Tang et al. (2023) considered sequential interactions between ML decisions and individuals, where they studied the impact of counterfactual fair predictions on statistical fairness but their goal is still to ensure parity-based fairness at the group level.

In this work, we focus on counterfactual fairness evaluated over individual future status (label), which accounts for the downstream effects of ML decisions on individuals. We aim to examine under what conditions and by what algorithms the disparity between individual future status in factual and counterfactual worlds can be mitigated after deploying ML decisions. To this end, we first introduce a new fairness notion called “lookahead counterfactual fairness (LCF)." Unlike the original counterfactual fairness proposed by Kusner et al. (2017) that requires the ML predictions received by individuals to be the same as those in the counterfactual world, LCF takes one step further by enforcing the individual future status (after responding to ML predictions) to be the same.

Given the definition of LCF, we then develop algorithms that learn ML models under LCF. To model the effects of ML decisions on individuals, we focus on scenarios where individuals subject to certain ML decisions adapt their behaviors strategically by increasing their chances of receiving favorable decisions; this can be mathematically formulated as modifying their features toward the direction of the gradient of the decision function (Rosenfeld et al., 2020; Xie & Zhang, 2024b). We first theoretically identify conditions under which an ML model can satisfy LCF, and then develop an algorithm for training ML models under LCF. We also extend the algorithm and theorems to path-dependent LCF, which only considers unfairness incurred by the causal effect from the sensitive attribute to the outcome along certain paths.

Our contributions can be summarized as follows:

• •

  We propose lookahead counterfactual fairness (LCF), a novel fairness notion that evaluates counterfactual fairness over individual future status (i.e., actual labels after responding to ML systems). Unlike the original CF notion that focuses on current ML predictions, LCF accounts for the subsequent impacts of ML decisions and aims to ensure fairness over individual actual future status. We also extend the definition to path-dependent LCF.

• •

  For scenarios where individuals respond to ML models by changing features toward the direction of the gradient of decision functions, we theoretically identify conditions under which an ML model can satisfy LCF. We further develop an algorithm for training ML models under LCF.

• •

  We conduct extensive experiments on both synthetic and real data to validate the proposed algorithm. Results show that compared to conventional counterfactual fair predictors, our method can improve disparity with respect to the individual actual future status.

Causal fairness has been explored in many aspects in recent years’ research. Kilbertus et al. (2017) point out that no observational criterion can distinguish scenarios determined by different causal mechanisms but have the same observational distribution. They propose the definition of unresolved discrimination and proxy discrimination based on the intuition that some of the paths from the sensitive attribute to the prediction can be acceptable. Nabi & Shpitser (2018) argue that a fair causal inference on the outcome can be obtained by solving a constrained optimization problem. These notions are based on defining constraints on the interventional distributions.

Counterfactual fairness (Kusner et al., 2017) requires the prediction on the target variable to have the same distribution in the factual world and counterfactual world. Many extensions of traditional statistical fairness notions such as Fair on Average Causal Effect (FACE) (can be regarded as counterfactual demographic parity) (Khademi et al., 2019), Fair on Average Causal Effect on the Treated (FACT) (can be regarded as counterfactual equalized odds) (Khademi et al., 2019), and CAPI fairness (counterfactual individual fairness) (Ehyaei et al., 2024) have been proposed. Path-specific counterfactual fairness (Chiappa, 2019) considered the path-specific causal effect. However, the notions are focused on fairness in static settings and do not consider the future effect. Most recent work about counterfactual fairness is about achieving counterfactual fairness in different applications, such as graph data (Wang et al., 2024a; b), medical LLMs (Poulain et al., 2024), or software debuging (Xiao et al., 2024). Some literature try to use counterfactual fairness for explanations (Goethals et al., 2024). Connecting counterfactual fairness with group fairness notions (Anthis & Veitch, 2024) or exploring counterfactual fairness with partial knowledge about the causal model (Shao et al., 2024; Pinto et al., 2024; Duong et al., 2024; Zhou et al., 2024) are also receiving much attention. Machado et al. (2024) propose an idea of interpretable counterfactual fairness by deriving counterfactuals with optimal transports (De Lara et al., 2024). While extending the definition of counterfactual fairness to include the downstream effects has been less focused on.

Several studies in the literature consider the downstream effect on fairness of ML predictions. There are two kinds of objectives in the study of downstream effects: ensuring fair predictions in the future or fair true status in the future. The two most related works to our paper are Hu & Zhang (2022) and Tang et al. (2023). Hu & Zhang (2022) consider the problem of ensuring ML predictions satisfy path-specific counterfactual fairness over time after interactions between individuals and an ML system. Tang et al. (2023) study the impact on the future true status of the ML predictions. Even though they considered the impact of a counterfactually fair predictor, their goal is to ensure parity-based fairness. Therefore, the current works lack the consideration of ensuring counterfactual fairness on the true label after the individual responds to the current ML prediction. Our paper is aimed at solving this problem.

Consider a supervised learning problem with a training dataset consisting of triples $(A,{X},Y)$, where $A\in\mathcal{A}$ is a sensitive attribute distinguishing individuals from multiple groups (e.g., race, gender), ${X}=[X_{1},X_{2},...,X_{d}]^{\mathrm{T}}\in\mathcal{X}$ is a $d$-dimensional feature vector, and $Y\in\mathcal{Y}\subseteq\mathbb{R}$ is the target variable indicating individual’s underlying status (e.g., $Y$ in lending identifies an applicant’s ability to repay the loan, $Y$ in healthcare may represent patients’ insulin spike level). The goal is to learn a predictor from training data that can predict $Y$ given inputs $A$ and ${X}$. Let $\hat{Y}$ denote the output of the predictor.

We assume $(A,{X},Y)$ is associated with a structural causal model (SCM) (Pearl et al., 2000) $\mathcal{M}=(V,U,F)$, where $V=(A,{X},Y)$ represents observable variables, $U$ includes unobservable (exogenous) variables that are not caused by any variable in $V$, and $F=\{f_{1},f_{2},\ldots,f_{d+2}\}$ is a set of $d+2$ functions called structural equations that determines how each observable variable is constructed. More precisely, we have the following structural equations,

where $pa_{i}\subseteq V$, $pa_{A}\subseteq V$ and $pa_{Y}\subseteq V$ are observable variables that are the parents of $X_{i}$, $A$, and $Y$, respectively. $U_{pa_{i}}\subseteq U$ are unobservable variables that are the parents of $X_{i}$. Similarly, we denote unobservable variables $U_{pa_{A}}\subseteq U$ and $U_{pa_{Y}}\subseteq U$ as the parents of $A$ and $Y$, respectively.

Theorem 4.1 below identifies more general scenarios under which LCF can be violated with a CF predictor.

This section introduces an algorithm for learning a predictor under LCF. In particular, we focus on a special case with the causal model and the individual response defined below.

Given sets of unobservable variables $U=\{U_{1},...,U_{d},U_{Y}\}$ and observable variables $\{X_{1},...,X_{d},A,Y\}$, we consider causal model with the following structural functions:

where $f_{i}$ is an invertible function⁴⁴4Several works in causal inference also consider invertible structural function, e.g., bijective causal models introduced in Nasr-Esfahany et al. (2023)., and $f_{Y}$ is invertible w.r.t. $U_{Y}$. After receiving the ML prediction $\hat{Y}$, the individual’s future features $X^{\prime}$ and status $Y^{\prime}$ change accordingly. Specifically, we consider scenarios where individual unobservable variables $U$ change based on the following

and the future attributes $X_{i}^{\prime}$ and status $Y^{\prime}$ also change accordingly, i.e.,

The above scenario implies that individuals respond to ML model by strategically moving features toward the direction that increases their chances of receiving favorable decisions, step size $\eta>0$ controls the magnitude of data change and can be interpreted as the effort budget individuals have on changing their data. Note that this type of response has been widely studied in strategic classification literature (Rosenfeld et al., 2020; Hardt et al., 2016a). The above process with $d=2$ is visualized in Figure 2.

The following theorem constructs a predictor $g$ that satisfies LCF, i.e., deploying the predictor $g$ in Theorem 5.1 ensures the future status $Y^{\prime}$ is counterfactually fair.

The above theorem implies that $g$ should be constructed based on the counterfactual random variable $\check{Y}$ and $U$. Even though $U$ is unobserved, it can be obtained from the inverse of structural equations. Quantity $T$ in Theorem 5.1 depends on the step size $\eta$ in individual response, and parameters $\alpha,\gamma,w$ in structural functions. When $p_{1}=\frac{T}{2}$, we can achieve perfect LCF.

It is worth noting that the results in Theorems 5.1 and 5.2 are for linear causal models. When the causal model is non-linear, it is hard to construct a model satisfying perfect LCF in Definition 4.1. Nonetheless, we can still show that it is possible to satisfy Relaxed LCF (Definition 5.2) for certain non-linear causal models. Theorem 5.3 below focuses on a special case when $X$ is not linearly dependent on $A$ and $U_{X}$ and it identifies the condition under which Relaxed LCF can be guaranteed.

Although the structural equation associated with $Y$ is still linear in $X$ and $U_{Y}$, we emphasize that such a linear assumption has been very common in the literature due to the complex nature of strategic classification Zhang et al. (2022); Liu et al. (2020); Bechavod et al. (2022). For instance, Bechavod et al. (2022) assumed the actual status of individuals is $Y=\beta X$, a linear function of features $X$. Zhang et al. (2022) assumed that $X$ itself may be non-linear in some underlying traits of the individuals, but the relationship between $X$ and $P(Y=1|X)$ is still linear. Indeed, due to the individual’s strategic response, conducting the theoretical analysis accounting for such responses can be highly challenging. Nonetheless, it is worthwhile extending LCF to non-linear settings and we leave this for future works.

An extension of counterfactual fairness called path-dependent fairness has been introduced in Kusner et al. (2017). In this section, we also want to introduce an extension of LCF called path-dependent LCF. We will also modify Algorithm 1 to satisfy path-dependent LCF.