FairIF: Boosting Fairness in Deep Learning via Influence Functions with Validation Set Sensitive Attributes

We consider the setting where each sample consists of an input $x\in\mathcal{X}$, a label $y\in\mathcal{Y}$, where $\mathcal{X}$ and $\mathcal{Y}$ are the input and output space respectively, and each example has a corresponding sensitive attribute $s\in\mathcal{S}$. For simplicity’s sake, assume that $\mathcal{S}=\{0,1\}$. Let $K$ denotes the number of classes, $[K]:=\{1,2,..,K\}$. We mainly focus on the classification problem and denote the classifier $h(x)=\operatorname*{arg\,max}_{i\in[K]}f^{i}_{\theta}(x)$, where $f_{\theta}\in\mathbb{R}^{K}$ is a neural network parameterized by $\theta\in\Theta$. We denote the number of parameters as $P$, then $\Theta\subseteq\mathbb{R}^{P}$. Denote the training data set of size $n$ as $\mathcal{D}=\{z_{1},z_{2},...,z_{n}\}$, where $z_{i}=(x_{i},y_{i})\in\mathcal{X}\times\mathcal{Y}$, and the validation data set (with sensitive attributes) of size $m$ as $\mathcal{D}^{s}=\{z^{s}_{1},z^{s}_{2},...,z^{s}_{m}\}$, where $z^{s}_{j}=(x_{j},y_{j},s_{j})\in\mathcal{X}\times\mathcal{Y}\times\mathcal{S}$. In this work, we are interested in the setting where the sensitive attribute $s$ is not available for the training set, as collecting large amount of data with the sensitive attributes is typically expensive and at the risk of privacy leakage (Yan et al., 2020). Instead, we assume that we have access to a small validation set with annotations of the sensitive attribute. The standard training procedure minimizes the empirical risk ${\mathcal{L}}(\mathcal{D},\theta)$,

where $\ell\left(z_{i},\theta\right):\mathcal{X}\times\mathcal{Y}\times\Theta\rightarrow\mathbb{R}^{+}$ is the loss function (e.g.,cross entropy loss function) for $z_{i}$. Our goal is to learn a fair neural network $f_{\theta^{\star}_{\bm{\epsilon}}}$ parameterized by $\theta^{\star}_{\bm{\epsilon}}\in\Theta$, through minimizing the weighted loss ${\mathcal{L}}_{\bm{\epsilon}}$:

where $\bm{\epsilon}=[\epsilon_{1},\epsilon_{2},...,\epsilon_{n}]^{\top}\in\mathbb{R}^{n}$ is a reweight vector of the training samples. Next, we introduce the fairness notions used in this work. For notation convenience, we denote the true positive rate on group 1 (sensitive attribute $s=1$) as $\text{TPR}^{(1)}=\mathbb{P}(h=1\mid s=1,y=1)$ where $h$ is the classifer’s prediction, and the true positive rate on group 0 (sensitive attribute $s=0$) as $\text{TPR}^{(0)}=\mathbb{P}(h=1\mid s=0,y=1)$. Similarly, we denote the true negative rate on group 1 as $\text{TNR}^{(1)}=\mathbb{P}(h=0\mid s=1,y=0)$ and the true negative rate on group 0 as $\text{TNR}^{(0)}=\mathbb{P}(h=0\mid s=0,y=0)$.

Accuracy Equality

requires the classification system to have equal misclassification rates across sensitive groups (Zafar et al., 2017a): $\mathbb{P}(h(x)=y|s=0)=\mathbb{P}(h(x)=y|s=1)$. Then, we define the Accuracy Difference (AD) as:

Equal Odds

is defined as $\mathbb{P}(h=1\mid s=1,y={{\color[rgb]{0,0,0}y^{\prime}}})=\mathbb{P}(h=1\mid s=0,y={{\color[rgb]{0,0,0}y^{\prime}),\forall y^{\prime}\in\{0,1\}}}$. It is sometimes also referred to as disparate mistreatment, aiming to equalize the true positive and false positive rates for a (binary-) classifier (Hardt et al., 2016). Following (Ozdayi et al., 2021; Yan et al., 2020), we define Average Odds Difference (AOD) as:

Equal Opportunity

is weaker than Equal Odds, but it typically allows for stronger utility (Hardt et al., 2016): $\mathbb{P}(h=1\mid s=1,y=1)=\mathbb{P}(h=1\mid s=0,y=1)$. Also, following (Ozdayi et al., 2021; Yan et al., 2020), we define Equality of Opportunity Difference (EOD) as:

The method of Influence Function (IF) (Koh and Liang, 2017) aims at approximating how the minimizer of the loss function $\theta^{\star}$ would change if we were to reweight the $i$-th training example. The key idea is to make a first-order approximation of change in $\theta^{\star}$ around $\epsilon_{i}=0$ with Taylor expansion. Specifically, if the $i$-th training sample is upweighted by a small $\epsilon_{i}$, then the perturbed risk minimizer $\theta^{\star}_{\epsilon_{i}}$ becomes:

The change of model parameters due to the introduction of the weight ${\epsilon_{i}}$ is:

where $H_{\theta^{\star}}=\frac{1}{n}\sum_{i=1}^{n}\nabla_{\theta}^{2}\ell\left(z_{i},\theta^{\star}\right)$ is the Hessian of the objective at $\theta^{\star}$, and $\mathcal{I}_{param}(z_{i})\in\mathbb{R}^{P}$ denotes the influence of sample $z_{i}$ on the model parameters. Following the assumption adopted in previous works (Koh and Liang, 2017; Koh et al., 2019; Barshan et al., 2020; Teso et al., 2021), $H_{\theta^{\star}}$ is positive definite. Note, this assumption is relatively weak, under the condition that $\theta^{\star}$ is the minimizer of the loss function. Similarly, the change in loss can be approximated (Barshan et al., 2020) as:

where the term $\mathcal{I}_{loss}(z_{i},z)\in\mathbb{R}$ represents the influence of sample $z_{i}$ on the loss computed over sample $z$.

The foundational insight underpinning FairIF  is that the influence of a training sample on the model prediction can be estimated through the influence function. This approach has been theoretically proven to offer precise estimations for linear models and has been empirically validated as effective in real-world applications (Koh and Liang, 2017; Koh et al., 2019; Zhang et al., 2022; Grosse et al., 2023). Specifically, For any continuously differentiable functions $F(\mathcal{D},\theta)\in\mathbb{R}$, $\mathcal{D}\subseteq\mathcal{X}\times\mathcal{Y}$, the change of $F$ with respect to $\epsilon_{i}$, the sample weight of $z_{i}$, can be computed based on the influence function:

The Equation (9) can be regarded as an extension of the conclusion of Equation (8). To apply the conclusion of Equation (9) to the fairness setting, we define the function $F$ as a general metric measuring fairness. Denote the two different groups of the validation set $\mathcal{D}^{s}$ as $\mathcal{D}^{0}$ and $\mathcal{D}^{1}$ respectively, where for any $z_{j}^{s}=(x_{j},y_{j},s_{j})\in\mathcal{D}^{0},s_{j}=0$ and $z_{j^{\prime}}=(x_{j^{\prime}},y_{j^{\prime}},s_{j^{\prime}})\in\mathcal{D}^{1},s_{j^{\prime}}=1$. Then, for the two groups, the change of function $F$ caused by permuting sample weights are,

and,

Then, the goal of achieving fairness is to achieve equalized performance $F$ over the two groups, i.e., solving $\mathbf{\epsilon}^{\star}$ to satisfy the following equation,

Combining Equations (10)(11)(12), we have:

Notably, the second term $F(\mathcal{D}^{0},\theta^{\star})-F(\mathcal{D}^{1},\theta^{\star})$ denotes the metric discrepancy of unweighted model across two demographic groups. We denote this empirically measurable discrepancy under metric $F$ as $\text{diff}(\mathcal{D}^{s},F,\theta^{\star})$. Then we have the following equation,

In order to find $\mathbf{\epsilon}^{\star}$ making $F(\mathcal{D}^{0},\theta^{\star}_{\mathbf{\epsilon}^{\star}})-F(\mathcal{D}^{1},\theta^{\star}_{\mathbf{\epsilon}^{\star}})=0$, we introduce the following optimization problem,

In the subsequent section, we will leverage this conclusion to address practical fairness challenges.

We now present FairIF, a straightforward two-stage methodology that eliminates the need for group annotations within the training set or alterations to the original models. Initially, we determine the sample weights using the influence function to ensure a balanced TPR and TNR performance across varied groups. Subsequently, in the second stage, we train the final model utilizing the reweighted training samples.

Stage One.

To address discrepancies across three fairness notions— AD, AOD, and AOE—FairIF  aims to equalize True Positive Rate (TPR) and True Negative Rate (TNR) between two groups by adjusting the sample weights, denoted as $\epsilon$. Section 5.1 offers an in-depth analysis of metric selection, highlighting that equalizing TPR and TNR serves as an effective fairness objective and can alleviate disparities under the aforementioned notions. However, since TPR and TNR are non-differentiable, they impede the use of Equation (13) to determine $\epsilon$. To circumvent this, we adopt the gumbel softmax technique (Jang et al., 2017) to approximate and render TPR and TNR differentiable. These approximations are represented as $F_{TPR}$ and $F_{TNR}$ respectively. Then, the objective can be defined as:

The first and second terms aim at balancing the TPR and TNR between groups with $\mathbf{\epsilon}$. And the last one is the regularization with weight factor $\lambda\in\mathbb{R}_{+}$, which enforces the weights close to zero as assumed by the influence function (Koh and Liang, 2017).

Stage Two.

Next, we train the final model $f_{\theta^{\star}_{\bm{\epsilon^{\star}}}}$ by reweighting the training samples with $\epsilon^{\star}$. The weighted loss is,

We present the detailed FairIF  training algorithm in Appendix B, Algorithm 1. Initially, the model is rigorously trained using standard empirical risk until reaching convergence, resulting in the parameter $\theta^{\star}$. Subsequent to this, we calculate the influence function and determine the appropriate weights to achieve equality under metrics TPR and TNR across groups in the validation set. In the final step, we train the model with reweighted data. Note, differently from previous methods (Ren et al., 2020; Teso et al., 2021) computing influence function on the fly, in our method, the influence function is computed after the model is converged, which not only saves the computation time of influence function, but also delivers a more accurate estimation. We leave the computation details of the influence function in Appendix C.

While Equation (14) effectively reduces disparity on the validation set, largely due to overparameterization (Zhang et al., 2021), its applicability to the testing phase and potential impact on classification performance remain questions. Subsequent sections, Section 5 and 6, address these concerns. Specifically, Section 5 demonstrates bounded disparities between validation and testing fairness metrics, and the task performance guarantee on the test set. Section 6, meanwhile, empirically evaluates FairIF  across eight datasets and six models.

In this section, we delve into an analytical examination underscoring the significance of balancing True Positive Rates (TPR) and True Negative Rates (TNR) across varied groups. The TPR and TNR equalization is pivotal in alleviating disparities in line with several fairness definitions. Consistent with the findings of prior research (Yu, 2021; Reddy et al., 2021), the ingrained bias in the models predominantly emerges from the following sources within the training data: 1. Variances in group sizes; 2. Discrepancies in class distribution within individual groups, often termed as the group distribution shift; 3. Inequalities in class sizes. To offer a clearer perspective on these bias forms, refer to Figure 1 where we have depicted a visualization of these three distinct bias categories.

With analysis of the relation between the three fairness notions and the TPR and TNR metrics, we derive the following proposition:

Proof sketch. Given the definitions of Average Odds Difference (AOD) and Equality of Opportunity Difference (EOD) detailed in Section 3.1, it becomes straightforward to deduce that both AOD and EOD would be zero if the TPR and TNR are harmonized between two distinct groups. Let’s delve into the Accuracy Difference (AD). It can be expressed as:

Here, $\mathbb{P}(y=1|s=0)$ and $\mathbb{P}(y=1|s=1)$ are represented by the variables $\alpha$ and $\beta$, respectively. Further dissecting the three forms of bias on a case-by-case basis, it becomes evident that AD either mirrors or is constrained by the combination of differences in TPR and TNR across these scenarios. The comprehensive proof of Proposition 1 has been catalogued for reference in Appendix A.

The aforementioned proposition demonstrates that by harmonizing the TPR and TNR across two groups, we can simultaneously achieve three pivotal notions of fairness, namely: Accuracy Equality, Equal Odds, and Equal Opportunity. This rationale supports our decision to employ a metric that combines both TPR and TNR within FairIF.

In the preceding section, we show that the metric combining TPR and TNR serves as an effective measure of fairness, and our proposed method aspires to ensure equality in TPR and TNR across distinct groups. In this section, our focus shifts to substantiating that the fairness performance remains consistent during testing. Before delving into the fairness assurances offered by FairIF, it’s essential to familiarize ourselves with the concept of Rademacher complexity(Bartlett et al., 2002), which measures the learnability of function classes. The Rademacher complexity for a function class is defined as below:

In what follows, we establish the bound for TPR disparity during test. A parallel argument can be made for TNR. As indicated in Section 3, the classifier is denoted by $h(x)$. However, diverging from prior notation for the sake of derivation convenience in this section, we modify the binary output of the classifier to range in $\{-1,1\}$. Its corresponding space, $\mathcal{H}$, encompasses hypotheses with values in $\{-1,1\}$. In order to align with the definition of Rademacher complexity, we represent the TPR disparity between two groups on dataset $\mathcal{D}$ as $r_{h}(\mathcal{D})$. Here, the TPR difference $r_{h}$ acts as a function dependent on the classifier function $h$. Consequently, the function class $\mathcal{F}$ is articulated as:

where $\mathcal{D}^{1,y=1}$ represent the positive samples from group 1 within $\mathcal{D}$, while $\mathcal{D}^{0,y=1}$ signifies the equivalent for group 0. From this definition, it’s clear that the range of $\mathcal{F}$ is bounded within $[0,1]$.

For simplicity in notation, we use $S$, rather than $\mathcal{D}^{S}$, to represent the validation set utilized by our method for sample weight computation. And the size of sample set $S$ is $m$. Then, let’s define $R(h)=\mathbbm{E}_{\mathcal{D}^{\text{test}}}[r_{h}(\mathcal{D}^{\text{test}})]$ as the TPR fairness risk of the classifier $h$ on the test set. Meanwhile, $\hat{R}_{S}(h)=\frac{1}{m}\sum_{z_{i}\in S}r_{h}(z_{i})$ represents the empirical fairness risk of the classifier $h$ on the sample set $S$. Note, samples within $S$ and $\mathcal{D}^{\text{test}}$ are i.i.d. and originate from the same data distribution. Based on the property of Rademacher complexity (Shalev-Shwartz and Ben-David, 2014), for any $\delta>0$, with probability at least $1-\delta$ over $S$:

Before we discuss the details of each terms of the previous inequality, let’s denote $S^{+}_{1}$ and $S^{+}_{0}$ as the positive samples in group 1 and group 0 in the sample set $S$ respectively. Then for the second term on the right-hand side (RHS) of Equation (17), we can express it in terms of $\widehat{\operatorname{Rad}}_{S}(\mathcal{H})$:

Our theoretical analysis elucidates key insights from Equation (17). Specifically, the second term on the right-hand side primarily reflects the Rademacher complexity of the model. This complexity is intricately connected to the family of neural networks to which the model belongs, and is constrained by moderate assumptions (Wei et al., 2018). Concurrently, due to the power of over-parameterization (Zhang et al., 2021), the first term, $\hat{R}_{S}(h)$, approaches zero (the discrepancy on the validation set can be optimized to nearly vanish). The third term’s magnitude correlates with the validation set size $m$, and as this size expands, our bound tightens. It’s noteworthy that for current deep learning datasets, a validation set, even if relatively small compared to the training set, is often ample. Empirical validations of this claim are further explored in Appendix G.

In this subsection, our primary objective is to understand the model’s task performance on the test dataset. To achieve this, we represent the test loss, using $\theta^{\star}_{{\epsilon}}$ for each sample, leveraging the first-order Taylor approximation:

For many benchmark datasets, $\|\epsilon\|^{2}\leq\frac{c}{n^{2}}$ holds (Yang et al., 2023), where $c$ represents the data quantity for which $\epsilon_{i}\neq 0$. Following the Influence Function methodology in (Koh and Liang, 2017), we omit the term $\mathcal{O}\left(\|\epsilon\|^{2}\right)$ in Equation (18). Expanding this further to encompass the test loss over the test set $\mathcal{D}^{\text{test}}$, we derive:

The concluding inequality assumes a positive real number $\gamma$ exists such that $\left\|\sum_{{z}_{i}\in{\mathcal{D}}}\mathcal{I}_{\text{param }}\left({z}_{i}\right)\right\|_{2}\leq\gamma$. Even though this assumption aligns with earlier works (Yang et al., 2023), it’s crucial to note that $\left\|\sum_{{z}_{i}\in{\mathcal{D}}}\mathcal{I}_{\text{param }}\left({z}_{i}\right)\right\|_{2}\leq\sum_{{z}_{i}\in{\mathcal{D}}}\left\|\mathcal{I}_{\text{param }}\left({z}_{i}\right)\right\|_{2}$. This term signifies the influence of each training sample on the test loss. Given that empirically introducing or excluding individual one training sample only marginally impacts the test loss, the assumption of $\gamma$ is validated. Moreover, with the objective pushing $\left\|\epsilon\right\|_{2}$ towards zero, variations in test performance between our fair model and the original unweighted model are constrained.

The synthetic dataset CI-MNIST (Reddy et al., 2021) is a variant of the MNIST dataset. In this dataset, each input image $x$ has a label $y\in\{0,1\}$ indicating odd or even respectively, and the background color, blue or red, is the sensitive attribute. CI-MNIST offers manual control over different types of bias in the dataset, such as the number of samples in each group and class, and the group distribution over the class. To examine the effectiveness of FairIF under different types of bias, we independently introduce the three major types of bias analyzed in Section 5 into the dataset. We compare FairIF with the five other baselines on three CI-MNIST variants based on three models: a multilayer perception (MLP), a convolutional network (CNN) and a LeNet(LeCun et al., 1998). The experimental results are shown in Table 1, Table 2 and Table 3 respectively. We report the results of the epochs with the best task accuracy performance. We provide the visualization of three different types of data bias in Figure 1.
Bias 1: Group Size Discrepancy. We set 15% of images in the two classes as red and the remaining as blue. The distributions over classes of each group are the same, and the number of images in each class is also the same. In this setting, the group of red background is under-representative.
Bias 2: Group Distribution Shift. We keep the amount of data within each class and each group to be the same, but set 85% of images in class 0 with blue background and 15% of images in class 1 with blue background. In this case, the group distributions over the classes are different.
Bias 3: Class Size Discrepancy. We set group distributions and the total amount of data within each group to be the same. And the amount of data in class 1 is 25% of class 0.

As shown in Table 1, 2 and 3, compared with the original model trained by ERM, FairIF can achieve lower discrepancies under three different notions of fairness, which demonstrates that FairIF can mitigate the fairness issue caused by the three different types of bias. In the meantime, we notice that FairIF mostly achieves higher accuracy than the original model under different biases, indicating the spurious correlation problem is alleviated by our sample reweighting mechanism. Also, we observe that the Influence baseline often achieves a high fairness level but at an unsatisfactory sacrifice of task performance, although we have conducted a hyperparameter grid search for the best fairness-utility trade-off. Further, comparing FairIF with all other state-of-the-art methods, we find that FairIF mostly has the top two performance on all the metrics regardless of the model and the dataset. This shows FairIF can achieve better fairness-utility trade-offs, even compared with methods directly using all the group information (e.g., FairSMOTE and DOMIND) and additional adversarial models (e.g., CFair and ARL).

In this study, we evaluate FairIF in comparison to five baseline models, utilizing three distinct tabular datasets: Adult (Asuncion and Newman, 2007), German (Asuncion and Newman, 2007), and COMPAS (Dieterich et al., 2016). To mitigate the risk of overfitting on these datasets, we follow the previous works (Li and Liu, 2022; Wang et al., 2022) and adopt the logistic regression for testing. Results can be found in Table 4. Furthermore, to demonstrate the effectiveness and scalability of our proposed method with large datasets and advanced models, we further tests on image datasets CelebA (Liu et al., 2015) and FairFace (Karkkainen and Joo, 2021) were conducted. For fairness, all baseline models were adapted using ResNet-18 as a feature extractor. It’s worth noting that results for the Influence baseline on image datasets were omitted due to prohibitive computational demands. These findings are presented in Table 5. Hyperparameters for all methodologies were tuned based on validation set performance.

As shown in Table 4 and 5, we find that just a small amount of group information on the validation set can allow FairIF to achieve low performance discrepancies between demographic groups. In the meantime, the accuracy performance of FairIF is very close to the original method, which supports our theoretical guarantees. Compared with the Influence baseline, FairIF often achieves a similar or better fairness level while maintaining better task performance on the three tabular datasets. And FairIF is compatible with deep neural networks on large-scale image datasets while the Influence baseline is not. Furthermore, even comparing with methods that directly use group annotation on the training set, e.g. CFair, DOMIND and FairSMOTE, FairIF can still most often outperform them in both accuracy and fairness metrics regardless of the backbone model, which proves its efficacy and scalability.

As observed in previous research (Wang et al., 2019), the pretrained models deliver discriminative results across different demographic groups, which might be caused by the pretraining procedure and the pretraining dataset collected from social networks, international online newspapers, and web searches. The proposed method FairIF yields fair models through changing the sample weights, which makes FairIF  a promising approach to remove the discrimination encoded in the pretrained parameters. Note, for methods requiring a modification of the network structure, the power of pretraining usually cannot be fully utilized. In this section, we aim to answer the question of how FairIF mitigates the fairness issue of the pretrained models. We evaluate FairIF with three commonly used pretrained models, ResNet-18, ResNet-34, and ResNet-50 (He et al., 2016). We finetune and evaluate these pretrained models on FairFace and CelebA and compare them with their counterparts trained with FairIF. The relative changes caused by using FairIF are presented in percentage in Table  6. With three different pretrained models on the two datasets, FairIF consistently mitigates the discrepancies of three different notions of fairness without hurting the accuracy performance.

We now probe into how FairIF achieves low performance discrepancies across different groups with the same or higher accuracy. In order to perform this analysis, we use the group annotations on the training data to closely examine what examples are up-weighted or down-weighted in the training set. Note, we don’t use the group information in the training stage. We show the samples mostly upweights and downweights by FairIF in Figure 2. For the CelebA dataset, we observe that FairIF tends to give more weight to examples of men with blond hair and white hair, while it decreases the weight of examples of females with blond hair. The result is as expected, the group (Male, Blond Hair) is the minority and the group (Female, Blond Hair) is the majority. For the FairFace dataset, FairIF  tends to upweight the samples of people with dark skin and downweight the samples of people with white skin. This is aligned with our expectation because the accuracy for group $s=1$ (white) is higher than the group $s=0$ (black). By upweighting the minority group and downweighting the majority group, the performance tends to balance. Note that FairIF computes different weights for different samples, and it is thus smarter than simply equalizing the weights of the samples from different groups in different classes in other reweighting methods such as FairSMOTE.