The Devil is in the Data: Learning Fair Graph Neural Networks
via Partial Knowledge Distillation

Fairness in the graph attracts increasing attention due to the superior performance of GNNs in different scenarios (Kipf and Welling, 2016; Hamilton et al., 2017; Xu et al., 2018). Commonly used fairness notions can be summarized as group fairness, individual fairness, and counterfactual fairness (Dwork et al., 2012; Kang et al., 2020; Ma et al., 2022). In this paper, we focus on group fairness which highlights that the model neither favors nor harms any demographic groups defined by the sensitive attribute. Efforts to improve group fairness have incorporated mitigating biases in graph-structured data (Current et al., 2022; Laclau et al., 2021) and constructing a fair GNNs learning framework (Agarwal et al., 2021). Mitigating data biases involves modifying the graph topology (i.e., adjacency matrix) (Li et al., 2021) and node attributes (Dong et al., 2022), which provides clean data for training GNNs. For example, FairDrop (Spinelli et al., 2021) proposes an edge dropout algorithm to reduce connection between nodes within the same demographic group. EDITS (Dong et al., 2022) modifies both graph topology and node attributes with an objective that minimizes distribution distance between several demographic groups. In the fair GNN learning framework construction, adversarial learning (Goodfellow et al., 2014) is a commonly used approach to learning a fair GNN for generating node representations or making decisions independent of the sensitive attribute. For example, FairGNN (Dai and Wang, 2021) utilizes an adversary to learn a fair GNN classifier which makes predictions independent of the sensitive attribute. FairVGNN (Wang et al., 2022) proposes mitigating the sensitive attribute leakage through generative adversarial debiasing. However, these two types of approaches make a strong assumption that the sensitive attribute is accessible, i.e., the sensitive attribute is known. Such an assumption may not hold in real-world scenarios due to legal restrictions. Although (Dai and Wang, 2021) studies fair GNNs in limited sensitive attributes, it still requires the sensitive attribute.

Different from previous works, this work aims to learn fair GNNs without accessing sensitive attributes. This requires novel techniques to overcome the challenges of learning in the absence of sensitive attributes. In addition, our proposed method, FairGKD, employs knowledge distillation to learn fair GNNs. It should be noted that (Dong et al., 2023b) has focused on addressing the fairness problem in GNN-based knowledge distillation frameworks by adding a learnable proxy of bias for the shallow student model. In contrast, FairGKD learns fair GNN students through a fairer teacher model, which is quite different from Dong et al.  (Dong et al., 2023b).

Since the legal and privacy limitations for accessing sensitive attributes, there are some progresses (Yan et al., 2020; Zhao et al., 2022a) to focus on fairness without demographic information in machine learning. For example, DRO (Hashimoto et al., 2018b) proposes a method based on distributionally robust optimization without access to demographics, which achieves fairness by improving the worst-case distribution. As the authors point out, DRO struggles to reduce the impact of noisy outliers, e.g., outliers from label noise. To avoid this impact, ARL (Lahoti et al., 2020b) improves fairness by addressing computationally-identifiable errors and proposes adversarially reweighted learning to improve the utility of worst-case groups. Inspired by label smoothing improving fairness, (Chai et al., 2022) utilizes knowledge distillation to generate soft labels instead of label smoothing. Although prior works improve fairness without demographics, these works design algorithms based on IID data, and their effectiveness in non-IID data, e.g., graph data, remains unknown. Thus, this work focuses on improving the fairness of GNNs working in graph-structured data without accessing sensitive attributes. Different from  (Chai et al., 2022), FairGKD aims to construct the fairer teacher and regards intermediate results (Romero et al., 2014) as soft targets.

For clarity in writing, consistent with prior works (Dai and Wang, 2021; Wang et al., 2022), we contextualize our proposed method and relevant proofs within the framework of a node classification task involving binary sensitive attributes and binary label settings. We represent an attributed graph by $\mathcal{G}=(\mathcal{V},\mathcal{E},\textbf{X})$ where $\mathcal{V}$ is a set of $\lvert\mathcal{V}\rvert=n$ nodes, $\mathcal{E}$ is a set of $\lvert\mathcal{E}\rvert=m$ edges. $\textbf{X}\in\mathbb{R}^{n\times d}$ is the node attribute matrix where $d$ is the node attribute dimension. $\overline{\textbf{X}}\in\mathbb{R}^{n\times d}$ is an all-one node attribute matrix. $S\in\{0,1\}^{n}$ represents the binary sensitive attribute. $\tilde{\textbf{X}}\in\mathbb{R}^{n\times(d-1)}$ is the node attribute matrix without the sensitive attribute. $\textbf{A}\in\{0,1\}^{n\times n}$ is the adjacency matrix. $\textbf{A}_{uv}=1$ represents that there exists edge $e_{uv}\in\mathcal{E}$ between the node $u$ and the node $v$, and $\textbf{A}_{uv}=0$ otherwise. For node $u$ and node $v$, if $S_{u}=S_{v}$, these two nodes are within the same demographic group. GNNs update the node representation vector $h$ through aggregating messages of its neighbors. As such, existing GNNs consist of two steps: (1) message propagation and aggregation; (2) node representation updating. Thus, the $k$-th layer of GNNs can be defined as:

where $\operatorname{AGGREGATE}^{(k)}(\cdot)$ and $\operatorname{UPDATE}^{(k)}(\cdot)$ represent aggregation function and update function in $k$-th layer, respectively. $\mathcal{N}(v)$ represents the set of nodes adjacent to node $v$.

We focus on group fairness which highlights the outputs of the model are not biased against any demographic groups. For group fairness, demographic parity (Dwork et al., 2012) and equal opportunity (Hardt et al., 2016) are two widely used evaluation metrics. In this paper, we utilize these two metrics to evaluate the fairness of models in the node classification task. Demographic parity (DP) requires that the prediction is independent of the sensitive attribute $S$ and equal opportunity (EO) requires the same true positive rate for each demographic group. Let $\hat{y}$ denote the node label prediction result of the classifier. $y\in\{0,1\}$ denotes the node label ground truth. The DP and EO difference between the two demographic groups can be defined as:

where small $\Delta_{DP}$ and $\Delta_{EO}$ imply fairer decision-making of GNNs.

This work aims to learn a fair GNN classifier $f_{g}(\cdot)$ which does not require accessing to demographics. With $\Delta_{DP}$ and $\Delta_{EO}$ as evaluation metrics, a fair GNN achieves minimum value for these two metrics. The problem of this paper can be formally defined as:

Problem Definition. Given a graph $\tilde{\mathcal{G}}=(\mathcal{V},\mathcal{E},\tilde{\textbf{X}})$, but non-accessible sensitive attributes, and partial node label $y$, learn a fair GNN $f_{g}$ for node classification task while maintaining utility.

In this subsection, we provide an overview of FairGKD, which is illustrated in Figure 3. FairGKD is motivated by our empirical observation on partial data training. The goal of FairGKD is to learn a fair GNN without accessing sensitive attributes. To achieve this, several issues need to be tackled: (1) improving fairness without accessing the sensitive attribute; (2) avoiding utility sacrifice resulting from partial data training; (3) achieving a trade-off between fairness and utility. To overcome the first two challenges, FairGKD employs the partial data training strategy to mitigate higher-level biases. Then, following the knowledge distillation paradigm (Romero et al., 2015; Hinton et al., 2015), FairGKD employs two fairness experts (i.e., models trained on only node attributes or topology) to construct a synthetic teacher for distilling fair and informative knowledge. To overcome the third challenge, FairGKD utilizes an adaptive optimization algorithm to balance loss terms of fairness and utility.

As shown in Figure 3 (a), FairGKD consists of a synthetic teacher and a GNN student model denoted by $f_{t}$ and $f_{s}$, respectively. $f_{s}$ is a GNN classifier for the node classification task, mimicking the output of $f_{t}$. The synthetic teacher $f_{t}$ aims to distill fair and informative knowledge $H$ for the student model. Specifically, $f_{t}$ is comprised of two fairness experts, $f_{tm}$ and $f_{tg}$, and a projector $f_{tp}$. Here, $f_{tm}$ and $f_{tg}$, which are trained on only node attributes and only topology, alleviate higher-level biases without requiring access to sensitive attributes. Due to partial data training, $f_{tm}$ and $f_{tg}$ may generate fair yet uninformative node representations denoted by $H_{tm}$, $H_{tg}$. To bridge this gap, the projector $f_{tp}$ is used to combine these uninformative representations and performs mapping to generate informative representation $H$. $H$ will be regarded as additional supervision to assist the learning of $f_{s}$. Mimicking fair and informative representation $H$, $f_{s}$ tends to generate fair node representation while preserving utility. During each training epoch, $f_{s}$ takes full graph-structured data $\tilde{\mathcal{G}}=(\mathcal{V},\mathcal{E},\tilde{\textbf{X}})$ as input and predicts node labels. With a trained and fixed synthetic teacher as the teacher, $f_{s}$ is supervised by node label $y$ to maintain utility and node representation $H$ from the synthetic teacher to improve fairness. To further achieve the trade-off between fairness and utility for $f_{s}$, an adaptive optimization algorithm (Zhao et al., 2022b) is employed to balance the training influence between the utility loss term (hard loss) and the knowledge distillation loss term (soft loss).

In summary, FairGKD is a demographic-agnostic framework that facilitates the learning of fair GNNs without sacrificing utility. Specifically, FairGKD is built upon knowledge distillation with partial data training. Different from the traditional teacher-student framework, FairGKD employs a synthetic teacher $f_{t}$, which is the combination of multiple models, to distill fair knowledge for learning fair GNNs. Benefiting from partial data training, FairGKD can learn fair GNNs without accessing sensitive attributes. Meanwhile, FairGKD can achieve fairness improvement for multiple sensitive attributes with just a single training session, as empirically demonstrated in Section 6.

In this subsection, we introduce the components of synthetic teacher $f_{t}$ and its optimization objective. The goal of $f_{t}$ is to distill fair and informative node representation $H$ for the student model. $f_{t}$ consists of two fairness experts $f_{tm}$, $f_{tg}$, and a projector $f_{tp}$. The MLP fairness expert $f_{tm}$ takes only node attributes without the sensitive attribute $\tilde{\textbf{X}}$ as input while outputting node representations $H_{tm}$. Meanwhile, the GNN fairness expert $f_{tg}$ takes graph $\overline{\mathcal{G}}=(\mathcal{V},\mathcal{E},\overline{\textbf{X}})$ with all-one node attributes matrix as input and outputs $H_{tg}$. Note that $\overline{\mathcal{G}}$ can be regarded as only graph topology due to the all-one node attributes matrix. The processing for obtaining the node representation $H_{tm}$ and $H_{tg}$ is as follows:

$f_{tm}$ and $f_{tg}$ are trained on partial data, i.e., only node attributes and only topology, respectively. According to observation in Section 4, $f_{tm}$ and $f_{tg}$ only inherit bias in node attributes or topology. As a result, $H_{tm}$, $H_{tg}$ are fairer node representations due to mitigating higher-level biases. However, these two representations may be uninformative due to partial data training of $f_{tm}$ and $f_{tg}$, which can lead to poor performance in downstream tasks. With such representations as the additional supervised information, it is challenging to train a fair student model that preserves utility. To address this issue, FairGKD employs a projector $f_{tp}$ to merge these two representations to generate fair and informative representations $H$. Specifically, $f_{tp}$ takes the concatenation of $H_{tm}$ and $H_{tg}$ as input and outputs $H$. This can be summarized as:

where $\oplus$ denotes the concatenate operation. Since partial data training mitigates higher-level biases, $H_{tm}$ and $H_{tg}$ generated by the trained model display more fairness. To maintain such fairness, $f_{tp}$ aims to seek bias neutralization between $H_{tm}$ and $H_{tg}$, which results in fair representations $H$. Eq. (8) combines the information from node attributes and graph topology while transforming $H_{tm}$ and $H_{tg}$ to a unified embedding space, similar to obtaining node representations with full data as input. As such, $f_{tp}$ can output informative yet fair representations $H$.

To ensure the implementation of the above design, FairGKD leverages a multi-step training scheme with a contrastive objective to train $f_{t}$. As shown in Figure 3(b), the multi-step training scheme can be divided into two steps, i.e., fairness experts training and projector training. (1) Fairness experts training aims to make fairness experts inherit less biases through partial data training. Given a full graph $\tilde{\mathcal{G}}=(\mathcal{V},\mathcal{E},\tilde{\textbf{X}})$ without the sensitive attribute, we have two partial versions of $\tilde{\mathcal{G}}$, i.e., $\tilde{\textbf{X}}$ and $\overline{\mathcal{G}}=(\mathcal{V},\mathcal{E},\overline{\textbf{X}})$. Then, $f_{tm}$, $f_{tg}$ generate node representations $H_{tm}$, $H_{tg}$ according to Eq. (6)-(7). Taking $H_{tm}$, $H_{tg}$ as input, the classifier, e.g., a linear classification layer, is employed to predict node labels $\hat{y}_{m}$, $\hat{y}_{g}$. Based on the predicted node labels and the ground truth, BCE function is utilized as the loss function to optimize weights of two fairness experts, respectively. (2) Projector training ensures the projector generates informative node representations through a contrastive objective and a trained GNN $f_{cg}$. Here, $f_{cg}$ has the same network structure as our GNN student but is directly trained on the full graph $\tilde{\mathcal{G}}$. After fairness experts training, we fix the model parameters of $f_{tm}$, $f_{tg}$ and generate $H_{tm}$, $H_{tg}$. As shown in Eq. (8), the projector takes the concatenation of $H_{tm}$, $H_{tg}$ as input and generates the node representations $H_{cg}^{{}^{\prime}}$. This process can be summarized as $H_{cg}^{{}^{\prime}}=f_{tp}(f_{tm}(\tilde{\textbf{X}})\oplus f_{tg}(\overline{\mathcal{G}}))$. With node representations $H_{cg}=f_{cg}(\tilde{\mathcal{G}})$ generated by $f_{cg}$ as the ground truth, we optimize the projector $f_{tp}$ by minimizing the contrastive objective. This maximizes the similarity between the representation of the same node in $H_{cg}^{{}^{\prime}}$ and $H_{cg}$, which means the synthetic model learns the informative node representations like $f_{cg}$.

For the contrastive objective, we follow the setting in Zhu et al. (Zhu et al., 2020). $h_{i}$ and $h_{i}^{{}^{\prime}}$ denote the node representation of node $i$ in $H_{cg}$ and $H_{cg}^{{}^{\prime}}$. $(h_{i},h_{i}^{{}^{\prime}})$ can be considered as a positive pair and the remaining ones are all negative pairs. Given a similarity function $sim(\cdot)$ to computing node representations similarity, the contrastive objective for any positive pair $(h_{i},h_{i}^{{}^{\prime}})$ can be defined as:

where $(h_{i},h_{j}^{{}^{\prime}})$ and $(h_{i},h_{j})$ are the negative pairs. $\tau$ is a scalar temperature parameter. We consider the similarity function $sim(\cdot)$ as $sim(h_{i},h_{i}^{{}^{\prime}})=\mathcal{D}(f(h_{i}),f(h_{i}^{{}^{\prime}}))$, where $\mathcal{D}$ is the consine similarity and $f(\cdot)$ is a projector. We compute the contrastive objective over all positive pairs $(h_{i},h_{i}^{{}^{\prime}})$ to obtain the overall contrastive objective:

With Eq. (10) as the objective function, the trained synthetic teacher can generate fair and informative node representations for guiding the student model. As shown in Theorem  ‣ A, we find that minimizing our contrastive objective is equivalent to maximizing the mutual information between $H_{cg}^{{}^{\prime}}$ and the original graph $\tilde{\mathcal{G}}$. This proves that the node representation generated by $f_{t}$ is informative. As a result, FairGKD learns fair and informative GNN students with a such teacher model.

The proof of Theorem  ‣ A is shown in Appendix A.

With a trained synthetic model $f_{t}$ as the teacher model, FairGKD trains a fair GNN student $f_{s}$. Following the knowledge distillation paradigm (Hinton et al., 2015), the GNN student learns to mimic the output of the synthetic teacher. Assuming that $f_{s}$ is a $k$-layer model with a single linear layer as the classifier, i.e., the first $k-1$ layer constitutes the GNN backbone, while the $k$-th layer serves as the classifier. Given a full graph $\tilde{\mathcal{G}}=(\mathcal{V},\mathcal{E},\tilde{\textbf{X}})$, $f_{s}$ generates node representations $\hat{H}=f_{s}^{(k-1)}(\tilde{\mathcal{G}})$ in ($k$-1)-th layer and predicts node labels $\hat{y}=f_{s}^{(k)}(\hat{H})$ in $k$-th layer. With two partial versions of $\tilde{\mathcal{G}}$ as input, the trained $f_{t}$ distills the knowledge $H=f_{t}(\tilde{\textbf{X}},\overline{\mathcal{G}})$ as the additional supervised information in soft loss. Node labels ground truth $Y$ is regarded as the ground truth in hard loss to make $f_{s}$ maintain high accuracy in the node classification task. Specifically, the objective function $L$ of training $f_{s}$ can be defined as:

where $L_{c}$ is the hard loss, which aims to maintain utility on the node classification task. We refer to $L_{c}$ as the binary cross-entropy function. $L_{kd}$ is the soft loss to optimize the GNN backbone for generating node representations similar to the output of $f_{t}$, which ensures that FairGKD learns fair GNNs. Similar to the optimization of the projector $f_{tp}$, we refer to $L_{kd}$ as the contrastive objective shown in Eq. (9), (10) during the knowledge distillation process. $\alpha$ and $\beta$, which are adaptive coefficients for balancing the influence of these two loss terms, are calculated by the adaptive algorithm shown in Section 5.4.

Due to the multi-loss of FairGKD, it can be challenging to achieve a balance between different loss terms, which indicates a trade-off between utility and fairness. To address this issue, an adaptive algorithm is used to calculate two adaptive coefficients (i.e., $\alpha$ and $\beta$) which balance the influence of the two loss terms. Inspired by the adaptive normalization loss in MTARD (Zhao et al., 2022b), we use this algorithm to balance $L_{c}$ and $L_{kd}$ instead of loss terms in multi-teacher knowledge distillation. The core idea behind this algorithm is to amplify the impact of the disadvantage term (i.e., loss term with slower reduction) on the overall loss function. In this regard, the disadvantage term will be assigned a larger coefficient and then contribute more to the overall model update. Specifically, let $L(t)$ denote the loss at epoch $t$, Eq. (12) can be formulated as:

We compute $\alpha(t)$ and $\beta(t)$ at each epoch $t$ through the loss decrease relative to the initial epoch, i.e., $t=0$. Here, we utilize the relative loss $\tilde{L}(t)=L(t)/L(0)$ to measure the loss decrease of loss $L$. A small value of $\tilde{L}(t)$ means a better optimization for the training model. Based on $\tilde{L}(t)$, $\alpha(t)$ and $\beta(t)$ can be computed by the following formulation:

where $lr$ is the learning rate of the hard loss term, $\gamma$ is a hyperparameter to enhance the disadvantaged loss which represents the loss term with a lower loss decrease.

In our scenarios, this algorithm suppresses the significantly decreasing loss term by assigning a smaller coefficient. Conversely, a bigger coefficient will promote the loss term that is slightly decreased. Meanwhile, $\alpha$ and $\beta$ are dynamically changing with model optimization. Thus, the GNN student is optimized by balancing two loss terms, resulting in the trade-off between utility and fairness. Additionally, we give a brief complexity analysis for our proposed method FairGKD in Appendix B.

With two different GNNs (GCN, GIN) as backbones, we compare FairGKD with three baseline methods on the node classification task. Specifically, we implement two FairGKD variants with the same network structure, i.e., FairGKD and FairGKD$\backslash$S, which represent training on graph-structured data with and without the sensitive attribute. Note that we denote the proposed method in Section 5 as FairGKD$\backslash$S for convenience. Table 2 presents the utility and fairness performance of two FairGKD variants and all baseline methods.

The following observations can be made from Table 2: (1) the proposed method FairGKD improves fairness while maintaining or even improving utility across all three datasets, highlighting the effectiveness of FairGKD in learning fair GNNs. (2) FairGKD achieves state-of-the-art performance in most cases. While it is not the best-performing method in some cases, FairGKD presents a small performance gap with the best result. This implies the superior performance of FairGKD on both fairness and utility. (3) FairGKD can improve the fairness of GNNs even without sensitive attribute information, and in some cases outperforms all baseline methods. This observation implies that FairGKD is a demographic-agnostic fairness method.

We also find that FairGKD$\backslash$S does not improve fairness on $Pokec$-$z$ and $Pokec$-$n$ dataset, but rather performs worse than Vanilla. However, comparing FairGKD$\backslash$S with Vanilla$\backslash$S trained on data without the sensitive attribute, we observe FairGKD$\backslash$S improves the fairness of GNNs. One potential reason could be that non-sensitive attributes, which are highly correlated to the sensitive attribute, leak sensitive information and are more prone to make the training GNN inherit bias. We also observe that FairGKD is effective for two commonly used GNNs. To further understand FairGKD, we visualize the node representation generated by the synthetic teacher and the trained GNN student of FairGKD on $Recidivism$ using t-SNE in Figures 4 and 5. Figure 4 shows indistinguishable node representations with respect to the sensitive attribute, which demonstrates fair knowledge distilled by the synthetic teacher and fair student learned by FairGKD. Figure 5 shows clear clustering of node representations with respect to the node label, indicating the superior utility performance of FairGKD.

We conduct ablations on datasets with the sensitive attribute. Specifically, we investigate how each of the following four components individually contributes to learning fair GNNs, i.e., fairness GNN expert, fairness MLP expert, the projector, and the adaptive optimization algorithm. We remove these four components separately and their results denoted by FairGKD$\backslash$G, FairGKD$\backslash$M, FairGKD$\backslash$P, and FairGKD$\backslash$A, respectively. For FairGKD$\backslash$G and FairGKD$\backslash$M, we regard the output of the remaining expert as the input of the projector. FairGKD$\backslash$P replaces the projector with mean operation. FairGKD$\backslash$A fixes $\alpha$ and $\beta$ in Eq. (12) as 0.5.

We show ablation results of FairGKD learning GCN student model on $Pokec$-$z$, and $Pokec$-$n$ datasets in Figure 6. We observe that FairGKD is stable in utility performance even if lacks some components. This is due to the fact that FairGKD trains the GNN student with full data as input and the knowledge distilling from the synthetic teacher is informative. The removal of any components, especially the fairness GNN expert and the fairness MLP expert, weakens the fairness improvement of FairGKD. This observation indicates that the synthetic teacher and the adaptive algorithm for loss coefficients are necessary for learning a fair GNN. Moreover, results of FairGKD$\backslash$A show higher $\Delta_{DP}$ value and standard deviations, which means worse fairness and more unstable performance. Thus, the adaptive algorithm for calculating loss coefficients is effective in achieving the trade-off between fairness and utility. Overall, the results show that all components of FairGKD are beneficial for the model fairness performance, while the adaptive algorithm also ensures the model fairness performance remains stable.

Due to the partial data training to mitigate higher-level biases, FairGKD can be employed in fairness scenarios involving various sensitive attributes within a single training session. To verify this, we conduct FairGKD with GCN as the student and evaluate the fairness performance of this student on different sensitive attributes. We study the fairness performance of FairGKD w.r.t. three sensitive attributes, i.e., region, gender, and age. Here, we set all sensitive attributes to be binary. We set age $\geq$ 25 as 1 and age $<$ 25 as 0.

The results of various sensitive attributes are shown in Table 3. We observe that a single implementation of FairGKD improves fairness under different sensitive attributes in most cases. This is due to the fact that FairGKD is not tailored for a specific sensitive attribute and mitigates higher-level biases. Due to such a design, FairGKD exhibits a weaker impact on fairness in sensitive attributes like gender. Moreover, we also find that vanilla outperforms FairGKD on utility performance by a small margin which can be ignored. Overall, experimental results demonstrate that FairGKD is a one-size-fits-all approach for improving fairness under different sensitive attributes.

We conduct hyperparameter studies of two parameters, i.e., $\tau$ and $\gamma$. Specifically, we only show the parameter sensitivity results on $Recidivism$ dataset due to similar observations on other datasets. Using GCN as the student model, we vary $\tau$ and $\gamma$ within the range of $\{$0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9$\}$ and $\{$0.001, 0.01, 0.1, 0.3, 0.5, 0.7, 1, 10, 100$\}$, respectively. As shown in Figure 7, we make the following observations: (1) the overall performance of FairGKD remains smooth despite the wide range of variation in $\gamma$ and $\tau$. Specifically, the performance of FairGKD is stable in terms of utility as well as fairness when $\gamma$ varies within the range of 0.001 to 10 or $\tau$ varies within the range of 0.1 to 0.9. (2) When $\gamma\geq 10$, FairGKD further improves fairness but the utility performance decreases rapidly. Due to a larger $\gamma$ strengthening the disadvantaged loss, this observation indicates that the soft loss $L_{kd}$ is the disadvantaged one in training. In this regard, a larger $\gamma$ improves fairness but sacrifices utility. To better achieve the trade-off between utility and fairness, a suitable $\gamma$ is necessary. (3) When $\tau\leq 0.1$, the fairness and utility performance of FairGKD both decrease. This suggests that the GNN student is not accurately replicating the output of the synthetic teacher. Overall, FairGKD remains stable with respect to the wide range variation of $\gamma$ and $\tau$. A suitable $\gamma$, e.g., $\gamma<10$, is beneficial for the trade-off between utility and fairness.