FedRN: Exploiting k-Reliable Neighbors Towards Robust Federated Learning

The key idea of \algname is to exploit useful information in other clients without violating privacy. Hence, the main challenge is to find the most helpful neighbor clients using limited information about each client. Intuitively speaking, these helpful neighbor clients have (1) high expertise on their data, or (2) data distributions similar to the target client’s:

• •

  Data Expertise: The training data should be sufficiently clean. The neighbor client with heavy noise could hinder identifying clean examples from the client’s noisy data.

• •

  Data Similarity: Data and class distributions should be similar to the target client’s. The distribution shift problem could lead to model incompatibility between target and neighbor clients.

We investigate how these two considerations affect selecting reliable neighbor clients for robust federated learning. Given a target client ${\sf c}$, a reliable neighbor client is defined to be one of the $k$-reliable neighbors $\mathcal{R}_{{\sf c}}(k)$, as described in Definition 3. Satisfying the condition, $k$-reliable neighbors can deliver clean and useful information even when the training data of the target client is unreliable due to the heavy label noise.

To compute the reliability score ${\rm R}({\sf c},{\sf n})$ in Eq. (3), we propose two indicators of data expertise and similarity using client’s local models. Since it is infeasible to access the raw data in neighbor clients, we leverage the server’s copies of local models, which are sent to the server during the model aggregation phase. By doing so, our approach does not compromise the privacy-preserving property in federated learning.

With the intention of selecting clients with cleaner examples, we define the data expertise score motivated by the memorization effect of DNNs. In general, DNNs memorize clean examples faster than noise examples during training, which has been observed consistently even at extreme noise ratios (Song et al., 2021). In this sense, the early memorization of clean examples leads to higher training accuracy if the client has cleaner local data. Consequently, during the model aggregation phase, the server receives the training accuracy of each client along with their local models, then computes and normalizes the data expertise Exp($\cdot$) per client,

where ${\rm Acc}$ denotes the training accuracy of the local model for the client ${\sf c}$; ${\rm maxAcc}$ and ${\rm minAcc}$ are the maximum and minimum training accuracy of the given client set.

Regarding the data similarity score, we propose to use predictive difference, which provides a hint to the similarity between two heterogeneous data distributions without the use of raw data. Hence, the data similarity between two clients is approximated by the prediction difference for the same random input, where a smaller difference indicates a higher similarity. Accordingly, after the local update phase, each client generates the same input $\tilde{x}$ of Gaussian random noise (Jeong et al., 2021) and transmits its softmax outputs $p(\tilde{x};\Theta_{{\sf n}})$ into the server, where $p(x;\Theta_{{\sf n}})$ denotes the softmax output of the client ${\sf n}$ for input $x$. The data similarity Sim($\cdot,\cdot$) between the target client ${\sf c}$ and neighbor client ${\sf n}$ is then computed by the server,

We use the cosine similarity denoted by Cosine to measure the predictive difference, because the symmetry of similarity is guaranteed, i.e., ${\rm Sim}({\sf c},{\sf n})={\rm Sim}({\sf n},{\sf c})$. The min-max normalization is also applied similarly to data expertise. We provide an in-depth analysis of how the two proposed indicators work precisely in Section 5.4.

For every communication round, each client participating in the upcoming round receives its $k$-reliable neighbors from the server. When updating the target client’s model, we follow the general philosophy of sample selection for robust learning by identifying clean examples from the noisy training data.

Different to robust methods in centralized scenarios, $k+1$ models, comprising $k$-reliable neighbors and the target client’s model, cooperate to mitigate the performance discrepancy among clients and obtain more reliable results. For each of the $k+1$ models, we fit two-component GMMs to model the loss distributions of clean and noisy examples in view of loss distributions being bi-modal (Li et al., 2020; Arazo et al., 2019). Based on the ensembled results of these GMMs, the target client’s model is only updated with examples selected as clean.

To be specific, our aim is to estimate the probability of each training example being clean with $k+1$ models with different reliability scores. At the beginning of the local update phase, the GMM is fitted to the loss of all available training examples in the target client by using the Expectation-Maximization (EM) algorithm. Given a noisy example $x$, the probability of being clean is obtained through its posterior probability for the clean (small-loss) modality,

where g denotes the Gaussian modality with a smaller mean (i.e., smaller loss) and $\ell(\cdot)$ is the loss function. Subsequently, the probability of each example to be clean is ensembled over $k+1$ models with their reliability scores ${\rm R}(\cdot,\cdot)$ in Eq. (3). Given a target client ${\sf c}$, the clean probability of joint mixtures can be formulated as:

The ensemble with the reliability scores give higher weight to the GMMs of trusted neighbors with high data expertise or similarity.

Lastly, \algname constructs a clean set $\mathcal{S}_{{\sf c}}$ such that each training example in the set has a clean probability greater than $0.5$,

where $\tilde{\mathcal{D}}_{{\sf c}}$ is the noisy data of the target client ${\sf c}$. The model is finally updated with only the clean set for a specified number of local epochs in the local update phase, while the complement of the set, which are likely to be noisy, are discarded for robust learning.

This collaborated approach with $k$-reliable neighbors considerably increases the robustness of low-performance clients with the aid of neighbors with high data expertise or similarity. Therefore, \algname reduces the performance gap between clients and enhances the overall robustness, thus leading to a stronger global model.

Due to the data heterogeneity in federated learning, the problem of distribution shift between clients’ local models is another challenge when ensembling their predictions for sample selection. Hence, \algname is integrated with the fine-tuning technique that helps quickly adapt to the local data distribution (Collins et al., 2021).

Before constructing the clean set in Eq. (8), we identify an auxiliary set of clean examples using only the target client’s model,

and fine-tune all the retrieved $k$-reliable neighbors for the auxiliary set. The auxiliary set could, however, be very noisy especially when the target client has heavy noise in its training data. To prevent feature extractors from being corrupted by such client-side noise, we only fine-tune the classification head. In Section 5.5, we examine the effect of the fine-tuning technique on data expertise and similarity used in neighbor search, and provide insights on their use for robust federated learning with noisy labels.

Algorithm 1 describes the overall procedure of \algname. We perform standard federated learning, FedAvg, for warm-up epochs before applying \algname, because robust training methods are generally applied after the warm-up phase in the literature (Han et al., 2018; Song et al., 2019). When training with \algname, retrieved neighbor models are fine-tuned for 1 epoch before sample selection (Line 11). Next, the target client identifies clean examples by leveraging $k$-reliable neighbor models (Line 13). The received global model is updated with estimated clean samples (Lines 14–15). In the aggregation phase, the central server aggregates the updated local models (Line 20).

The main additional cost of \algname is the communication overhead from sending $k$-reliable models from the server to the selected client (see Appendix 5.9 for detailed cost analysis). However, a small number of reliable neighbors is sufficient (see Section 5.6) and the cost can be drastically reduced by over a factor of 32 times in modern federated learning by sending the difference of parameters (Jeong et al., 2021) or a compressed model (Malekijoo et al., 2021). Furthermore, there is no issues in model convergence because we applied a sample selection approach, which don’t has convergence problems, in a decentralized setting using FedAvg (McMahan et al., 2017), which has proven model convergence.

We verified the superiority of \algname compared with five robust learning methods on three real-world or synthetic benchmark datasets in a Non-IID federated learning setting, where the Non-IID setting is more realistic and difficult than the IID setting (Zhu et al., 2021; Zhao et al., 2018; Briggs et al., 2020).

For the federated learning setup with noisy labels, we merged the standard setting in the two research communities:

• •

  Non-IID Data: Two popular ways of client data partitioning were applied (Zhu et al., 2021); (i) Sharding: training data is sorted by its class labels and divided into “$S\!\times\!(\#~{}{\rm clients})$” shards, which are randomly assigned to each client with an equal number of $S$. (ii) Dirichlet Distribution: each client is assigned with training data drawn from the Dirichlet distribution with a concentration parameter $\beta$. For the experiments, $S$ was set to $2$, $5$, $10$, or $20$, and $\beta$ was set to $0.5$.

• •

  Label Noise: We injected artificial noise by following typical protocols (Song et al., 2022); (i) Symmetric Noise: a true label is flipped into all possible labels with equal probability. (ii) Asymmetric Noise: a true label is flipped into only a certain label. Note that we injected varying levels of label noise into the clients, so each client had different noise rates. For instance, a noise of $0.0$–$0.8$ means that the noise rate increases linearly from $0.0$ to $0.8$ as the client’s index increases; thus, the average noise rate of clients’ training data is $0.4$. We further tested for mixed noise where the two different types of label noise are injected into the clients. That is, symmetric noise is applied to half of the clients, while asymmetric noise is applied to the remaining clients.

We did not inject any label noise into mini-WebVision because it contains real label noise whose rate is estimated at $20.0\%$ (Song et al., 2022).

As for hyperparameters related to our method, \algname requires the balancing term $\alpha$ and number of reliable neighbors $k$. We used $\alpha=0.6$, which was obtained via a grid search (see the section 5.7). We used $k=1$ and $k=2$ since the performance gain of \algname was consistently high as long as $k\geq 1$, which is detailed in Section 5.6.

The hyperparameters of the baseline robust methods were configured favorably in line with the literature. We clarify the hyperparmeter setup of all baseline algorithms, as follows:

• •

  Joint Optimization (Tanaka et al., 2018) requires the two coefficients for its two regularization losses for robust learning. These two coefficients were set to be 1.2 and 0.8, respectively.

• •

  SELFIE (Song et al., 2019) requires the uncertainty threshold and the length of label history as its hyperparameters. As suggested by the authors, they were set to be 0.05 and 15, respectively.

• •

  DivideMix (Li et al., 2020) applies MixMatch for semi-supervised learning, which requires the sharpening temperature, the unsupervised loss coefficient, and the Beta distribution parameter. They were set to be 0.5, 25, and 4, respectively.

• •

  Robust FL (Yang et al., 2022) requires two coefficients for its two additional regularization losses similar to Joint Optimization. They were set to be 1.0 and 0.8, respectively.

All of the algorithms were implemented using PyTorch $1.7.0$ and executed using four NVIDIA RTX 2080Ti GPUs. In support of reliable evaluation, we repeated every task thrice and reported the average test (or validation) accuracy, which is the common measure of robustness to noisy labels (Han et al., 2018; Chen et al., 2019).

All methods except FedAvg and Joint Optimization follow the pipeline of learning with sample selection. Hence, we evaluate the sample selection performance of them using the two metrics, namely label precision (LP) and recall (LR), which respectively represent the quality and quantity of examples selected as clean (Han et al., 2018; Song et al., 2021),

empty

where $\mathcal{D}_{\sf c}$ and $\mathcal{S}_{\sf c}$ denote the training data and its selected clean set of the client ${\sf c}$, respectively.

Figure 3 shows the LP and LR curves averaged across all the clients per epoch on CIFAR-10 with symmetric and asymmetric label noise. The results of Robust FL are omitted due to its too low label precision and recall under the Non-IID scenario. After the warm-up period of $100$ transmission rounds, \algname shows the highest LP and LR with a large improvement of up to $0.12$ and $0.11$, respectively. In addition, its dominance over other robust learning methods is consistent regardless of the noise type. Therefore, these results indicate that the superior robustness of \algname in Tables 2–4 is due to its high LP and LR.

We justify the use of two indicators for data expertise and similarity in Eqs. (4) and (5).

• •

  Data Expertise: Figure 4 shows the training convergence rate (training accuracy) of all clients sorted in ascending order by their noise rate. Owing to the memorization effect of DNNs, the training accuracy of the local model indeed has a strong correlation of over $0.93$ with respect to varying levels of label noise. These results confirm that the client’s training accuracy represents its data expertise.

• •

  Data Similarity: Figure 5 shows similarity matrices between client’s softmax outputs for a Gaussian random noise when trained on data with or without label noise. As clients with similar Ids (indices) have similar data distributions in our Non-IID setting, the similarity around diagonal entries is high, as can be seen from Figure 5(a). Even with label noise shown in Figure 5(b), it turns out that the similarity trend remains. Therefore, it can be concluded that the similarity between the softmax output is a robust metric to find clients with similar data distributions.

These empirical studies provide empirical evidence for the use of our two indicators, ${\rm Exp}(\cdot)$ and ${\rm Sim}(\cdot,\cdot)$.

We analyze the effect of fine-tuning on data expertise and similarity used in neighbor search. Table 5 summarizes the performance of \algname ($k=2$) with and without fine-tuning, where a small $\alpha$ indicates that data similarity is considered more than data expertise. When fine-tuning is deactivated, the benefit of retrieving neighbors with high data similarity is remarkably dominant, because lower data similarity implies a larger distribution shift between the target and neighbor models. On the other hand, an opposite trend is observed when fine-tuning is activated. The test accuracy is high when data expertise is properly considered in conjunction with data similarity. Overall, fine-tuning does not only help mitigate the distribution shift problem but also significantly improves performance of \algname.

A larger number of reliable neighbors could increase the communication overhead in federated learning. We, therefore, investigated the change in performance of the global model according to the the number of reliable neighbors. Tables 6 and 7 show the classification accuracy on CIFAR-10 and CIFAR-100 respectively with varying number of neighbors. The results show that \algname with $k=1$ and $k=2$ generally shows comparable performance than $k>2$; there is little improvement in performance with the increase in the number of reliable neighbors as long as $k>1$. In other words, using only one or two reliable neighbors is sufficient to improve the robustness, because the proposed reliability score successfully retrieves the most helpful neighbor client among all the neighbors. Therefore, in a scenario where the communication burden is significant, practitioners may reduce the communication overhead by only using a single reliable neighbor with \algname to handle the problem of noisy labels.

We adjusted $\alpha$ to determine the contribution of data expertise and similarity in neighbor search. To ascertain the optimal value, we repeated the experiment on CIFAR-10 with various $\alpha$ values, as summarized in Table 10. Because the fine-tuning technique is incorporated in \algname, merging data expertise and similarity provides a synergistic enhancement in general. In particular, when $\alpha=0.6$, the performance gain is the highest on average. Based on these results, we set $\alpha$ to $0.6$ for all experiments in Section 5.

We analyzed the usefulness of $k$-reliable neighbors compared with $k$-random neighbors. For $k$-random neighbors, we replaced Lines 6 and 10 of Algorithm 1 with random client selection. Table 9 summarizes the test accuracy of \algname with $k$-reliable and $k$-random neighbors. Since our fine-tuning technique was incorporated with $k$-random neighbors, using $k$-random neighbors improves the robustness against noisy labels, but \algname with $k$-reliable neighbors maintains its dominance in every case. In detail, if the neighbors with low data expertise are selected, they misclassify mislabeled examples as clean examples in sample selection. Likewise, the neighbors with significantly different data distributions harm identifying clean examples from the target client’s noisy data due to the distribution shift problem. It can be confirmed experimentally that this issue becomes more prominent when the noise rate is high. In conclusion, we should leverage the reliability of neighbors to obtain a global model robust to noisy labels.

We analyze the communication and computation cost of \algname compared with other six methods, as summarized in Table 8. In this comparison, the communication cost is split into two parts, “server$\rightarrow$client” and “client$\rightarrow$server,” while the computation cost is split into two other parts, “forward” and “backward,” and the “total cost” aggregates all the costs for each perspective.

• •

  Communication Cost: \algname requires 3M or 4M communication cost in total when using one or two reliable neighbors, which is a sufficient number for general use cases. The remaining communication cost for calculating data similarity and expertise (i.e., $Acc({\sf c})$ and $p(\tilde{x};\Theta_{\sf c})$) is negligible. By contrast, Co-Teaching and DivideMix both require 4M communication cost in total because they maintain two models for co-training. In summary, \algname leads to greater performance with comparable or lower communication cost compared with other baseline methods.

• •

  Computation Cost: We exclude the cost of fitting GMMs and searching $k$-reliable neighbors because their costs are negligible compared to those of the forward and backward steps. Along the pipeline of our local update phase, \algname first requires $k$F and $k$B computation cost to perform fine-tuning with $k$-reliable neighbors for one epoch. Next, to construct the clean set, $(k+1)$F communication cost is needed due to the inference with $k+1$ available models. As the constructed clean set is deterministic over all the local epochs, only $e$F+$e$B computation cost is required to train a single model for the given local epoch $e$ in each communication round. Consequently, \algname needs $(e+2k+1)$F + $(k+e)$B computation cost in total. In a typical deep learning pipeline, the cost of the backward step is relatively smaller than that of the forward step. Hence, the computation cost for the forward step is the major issue. In light of this fact, the forward cost of \algname is comparable to those of FedAvg, Joint-optim, and SELFIE, and less than those of Co-teaching and DivideMix, because the number of local epochs is larger than the number of reliable neighbors, $e>k$. Among the compared methods, DivideMix is the slowest method in that it performs $m$ data augmentations with two models for each local epoch, so its forward cost is $(4m+2)e$F.

Therefore, with one or two reliable neighbors, \algname greatly improves the robustness of the global model without adding much communication and computation cost in the scenario of federated learning with noisy labels.