FedCC: Robust federated learning against model poisoning attacks

FL is vulnerable to poisoning attacks due to its distributed nature and the server’s inability to examine dataset quality and model parameter integrity directly. Model poisoning is particularly destructive among various poisoning attacks, as adversaries stealthily manipulate local model parameters to maximize overall performance damage [3, 5]. Thus, this paper focuses on model poisoning attacks.

Model poisoning attacks allow client-side adversaries to alter local model parameters before sending them to the server. To avoid the global model from divergence, attackers optimize the compromised local model for both training loss and an adversarial objective, enhancing its stealthiness [5, 8]. They can also adjust model hyperparameters, such as the learning rate, local epochs, batch size, and optimization objective, to manipulate training rules before and during local training.

Based on the attacker’s goals, model poisoning attacks can be classified as untargeted or targeted. Untargeted attacks aim to degrade overall global test accuracy by manipulating compromised local models. In contrast, targeted attacks aim to mislead the model into misclassifying specific inputs, leaving other classes unaffected. This paper introduces a backdoor ability into targeted model poisoning attacks, where inputs belonging to a chosen class are intentionally misclassified to a chosen label, whereas the performance of other classes remains unaffected. Its added stealthiness makes the attack more challenging to detect [23, 21].

Several Byzantine-robust aggregations based on summary statistics or anomaly detection have been proposed to mitigate model poisoning and backdoor attacks in FL. For example, Krum [4] selected the update with the smallest Euclidean distance to the remaining updates as the new global model. Still, it ignored outliers and assumed IID data only [2]. Median [26] selects the coordinate-wise median of updates, also ignoring outliers. Trimmed Mean [26] excludes extreme values before averaging, and Bulyan [11] combines Krum and Trimmed Mean for selection. These methods rely on knowing the number of attackers, which is often unavailable in practice.

Inspired by the fact that Sybil’s crafted model parameter’s directions are more similar to each other than to benign ones, Foolsgold [9] identified Sybils by measuring cosine similarity between local model gradients, marking the most similar ones as Sybils. However, it struggled with a single malicious client when there were no other malicious clients for comparison. Additionally, this approach may overfit non-IID data, ironically leading to suboptimal performance with IID data. FLTrust [6] used trust scores based on ReLU-clipped cosine similarities to weigh local clients’ parameters differently but risked privacy by requiring clients to share raw data with the server. Furthermore, precise weight scaling is paramount to ensure model convergence.

Previous works often used summary statistics (mean or median), direction (cosine similarity), or distance (Euclidean) of weight vectors, but they struggled to effectively distinguish between benign clients with non-IID data and malicious clients, often misclassifying benign clients as malicious. The lack of performance might be partially attributed to the inadequacy of Euclidean geometry as a metric to compare neural network representations. In contrast, our proposed approach utilizes kernel-based metric (Kernel CKA) to measure the similarity between global and local models, reliably identifying compromised model parameters among the mixture of clients and enhancing defense against model poisoning attacks in FL.

The penultimate layer is the second-to-last layer (before the softmax layer) of a neural network model, as illustrated in Figure 1. Wang et al. [22] discovered that PLRs are effective in distinguishing malicious models from benign ones; within benign clients, benign PLRs follow the same distribution, whereas malicious PLRs do not, across multiple datasets and neural network architectures. Specifically, they demonstrated that distances among benign PLRs are smaller than those of benign and malicious PLRs. Their proposed method, FLARE, assigned a trust score to each client based on pairwise PLR discrepancies defined by Maximum Mean Discrepancy (MMD) among all model updates, allocating lower values to those farther from the benign distribution. The model updates are then scaled and averaged, weighted by each client’s trust score. However, FLARE continuously redirects the global model using server-owned raw data, increasing the risk of data leakage and jeopardizing data privacy.

Centered Kernel Alignment (CKA) [13] is a highly accurate similarity metric that measures how similar two differently initialized or trained neural networks are, producing a value between 0 (not similar) and 1 (identical). CKA can be calculated by:

where $HSIC$ is Hilbert-Schmidt Independence Criterion [10] (i.e., a not normalized version of CKA) and $K$ is the RBF kernel. It constructs similarity kernel matrices in the weight space and compares these matrices to characterize the representation space, allowing comparisons between layers with varying widths and different initialization schemes.

CKA includes several important properties for similarity measures of neural networks. It is non-invariant to invertible linear transformations, meaning similarity scores change if such transformations are applied. This property is vital; the gradient descent algorithm is not invariant to these transformations because similarity metrics invariant to linear transformations are inaccurate on models trained with gradient descent. It is also invariant to orthogonal transformations and isotropic scaling, which are desirable for neural networks trained with gradient descent owing to their stochastic nature. These properties make CKA a precise and reliable metric for measuring similarities between neural networks, particularly when comparing potential models from malicious clients.

CKA has been utilized to address data heterogeneity issues, taking into account that the similarity of non-IID data is notably lower than that of IID data [20], particularly in specific layers [19]. However, CKA has yet to be leveraged as a defense mechanism against attacks involving models from malicious clients where the data are distributed across the clients in a non-IID manner.

We consider two primary attack strategies in the context of FL: untargeted model poisoning and targeted backdoor attacks. Untargeted attacks (Fang attacks [8]) aimed to evade robust aggregation rules like Krum and Coordinate-wise Median or Trimmed Mean. Attackers manipulate the model parameters to degrade the global model’s overall accuracy. Targeted backdoor attacks, represented by Bhagoji et al. [3], aim to deceive the global model into misclassifying specific data samples the attacker chose. The objective is to assign a different label chosen by the attacker while maintaining high accuracy for the remaining classes, making the attack more inconspicuous.

• •

  An attacker on the client side can control multiple compromised clients.

• •

  An attacker has full control over at most $k<n/2$ clients.

• •

  An attacker has knowledge about compromised clients’ data samples, current local model parameters, global model parameters of the previous round, and hyperparameters, such as learning rate, optimization method, loss function, batch size, and the number of local epochs.

• •

  An attacker has no knowledge or control over the server and honest clients.

Figure 2 depicts an overview of FedCC. (1) A server initializes and advertises a global model to $n$ participating clients, selected out of total $k$ clients with a probability of fraction $C$. (2) The clients individually train their local models using their respective datasets and send the trained weights to the server for aggregation. Our proposed method intervenes at this moment. (3) The server extracts PLRs from the global model and each local model, and (4) compares RBF Kernel CKA values between them. (5) The CKA values are then clustered into two, potentially representing benign and malicious clients. (6) Since the server cannot assure whether the clients with higher similarities are malicious, it performs a coordinate-wise average of the weights from clients belonging to the cluster with a larger number of members, backed by an assumption that the number of malicious clients does not exceed $n/2-1$. (7) Finally, the server updates the global model for the current epoch and proceeds to the next iteration by distributing the updated global model to newly selected $n$ clients.

When filtering malicious clients in the presence of non-IID data, a major challenge is the server’s lack of knowledge regarding the data distribution. Specifically, even if the data is benign, local models can exhibit significant angular or magnitude differences if the data is non-IID. We thus require a similarity metric independent of data distribution and not influenced by the distance or direction of model parameters.

Penultimate Layer is the output of the second-to-last layer before the softmax layer in CNN. In federated learning, where the server cannot access local training data, it receives trained model parameters (weights) from selected clients. Therefore, the collected weights are the only available information for determining whether a client is malicious. PLR takes the end result of the convolution and pooling process that will reach a classification decision. Thus, we assumed that computed weights of the second last layer might contain the essential information that strongly influences classification results.

To justify the utilization of PLRs, we measure the distance between two clusters of clients (benign and malicious) using dendrograms with a single linkage metric and correlation distance method. Figure 3 presents cluster distances of each layer under different attack scenarios, such as untargeted-mKrum attacks in IID or non-IID settings. Observably, the cluster distance is the most considerable in PLR to other layers, indicating the highest degree of differences. Based on this observation, we conclude that PLR contains the most distinguishable and indicative information. Therefore, we utilize the weights of the penultimate layer to compare their similarity to the global model.

Kernel CKA[13] is a similarity metric to assess the similarity between two neural networks. It compares representations at different layers or models trained in various ways. The sophisticated similarity comparison ability of Kernel CKA makes it a suitable choice for our purpose.

To justify the use of Kernel CKA, we compared its performance with other similarity metrics such as linear CKA, MMD (not normalized CKA), cosine (angle), and Euclidean similarity (distance). We evaluated the test accuracy of FedCC using the CIFAR10 dataset under two untargeted attacks and one targeted attack, separately in IID and non-IID settings. The results are summarized in Table I. In the ’targeted’ column, the values represent the combination of test accuracy and backdoor confidence, with higher test accuracy and lower backdoor confidence indicating better performance. We observed that Kernel CKA consistently yielded the highest performance across multiple experiments. Therefore, we decided to utilize Kernel CKA to compare PLR similarities.

FedCC is an aggregation method that combines selected weights without scaling, based on the measurement of Kernel CKA similarities between the global and local models’ PLRs. A pseudo algorithm is described in Algorithm 1. A server first extracts PLRs of the global and local models and calculates the kernel CKA between them. We specifically focus on comparing PLRs because they exhibit the most distinguishable differences among differently trained models, as demonstrated in previous sections. Additionally, we utilize RBF Kernel CKA, as it effectively captures similarity differences between non-IID and malicious clients. The similarity values obtained are then clustered using a simple K-means clustering algorithm. Any clustering method that produces only two clusters can be used. Considering the assumption that the number of attackers is less than half of the total number of clients ($n/2$), we designate the clients in the cluster with fewer members as suspects. The indices of these suspicious clients are stored in a list called ‘suspects.’ We proceed by selectively averaging the weights of clients whose index does not belong to the ‘suspects’ list, excluding the weights of the suspicious clients from the aggregation. Note that the weights are averaged without scaling, as scaling could impede the model from convergence even when the loss reaches optima. We named our proposed method FedCC.

Compared to other methods such as Krum, Bulyan, and FLARE, which either require prior knowledge of the number of attackers or allow the server to have access to raw data, FedCC does not rely on any information about the clients or any instances of data. This ensures strong privacy preservation within the federated learning framework. Furthermore, FedCC effectively distinguishes between malicious and benign clients, even in practical but challenging scenarios where benign clients have non-IID data.

We use three benchmark vision datasets: Fashion-MNIST (fMNIST) [24], CIFAR10, and CIFAR100 [14].

fMNIST consists of 60,000 training and 10,000 testing samples of 28$\times$28 grayscale images and associating labels of 10 classes. The labels include T-shirt/top, Trouser, Pullover, Dress, Coat, Sandal, Shirt, Sneaker, Bag, and Ankle boot.

CIFAR10 consists of 50,000 training and 10,000 testing examples of 32$\times$32 color images and associating labels of 10 classes. The classes include airplane, automobile, bird, cat, deer, dog, frog, horse, ship, and truck.

CIFAR100 is the same as CIFAR10 except for the categorization. It is labeled as 100 classes, each containing 600 images. Thus, there are 500 training and 100 testing images per class. Each class (i.e., ‘fine’ label) belongs to one of the 20 superclasses.

Non-IID is a more feasible assumption considering the diverse and massive nature of data in practice. To standardize non-IID settings, we use Dirichlet distribution, which is well-known for its capability of representing real-world data [15]. A concentration parameter $\alpha$ controls the degree of non-IID; the smaller $\alpha$ is, the more likely the clients hold examples from only one randomly chosen class.

Considering that the end devices are normally incapable of handling heavy computation due to resource or communication constraints, we used lightweight CNN architectures for the experiments. For fMNIST, we used a 4-layer-CNN with two convolutions and two fully connected (FC) layers, incorporating dropout with probabilities of 0.25 and 0.5 before and after the FC layer, respectively. For CIFAR10, we utilized a 5-layer-CNN with three convolutions, each followed by a max-pool layer and two FC layers, applying dropout with a probability of 0.5 before the first FC layer. Finally, for CIFAR100, we adopted the LeNet architecture. Table VI, Table VII, and Table VIII in Appendix summarize model architectures for each benchmark dataset.

We implement two untargeted attacks in [8] and one targeted attack [3], defense baselines, including Krum [4], Coomed [26], Bulyan [11], and FLARE [22], and FedCC. We have ten clients with a client participation fraction of 1.0 unless stated otherwise. The number of attackers is 30% of the total clients (three) in untargeted and 10% (one) in targeted attacks as in [22]. We also assumed all clients have the same NN architecture. Under the targeted backdoor attack, benign clients go through three, whereas compromised clients undergo six local epochs. We train each client’s local model using an Adam optimizer with a learning rate of 0.001, following the experiment in [22]. All results are averaged after running each experiment three times.

We assessed the performance of our defense mechanism using two metrics: backdoor confidence (confidence) and global model test accuracy (accuracy). The confidence indicates the likelihood of misclassification, with values ranging from 0 to 1 (lower values indicating better defense performance). Accuracy measures the overall test accuracy of the global model when various defense methods are applied when attacks exist, with values ranging from 0 to 100 (higher values indicating better performance). We reported both metrics in the presence of backdoor attacks while focusing solely on accuracy for untargeted attacks—as the attacker’s goal is to deteriorate the overall accuracy.

Following the aforementioned non-IID simulation, we set $\alpha$ to 0.2 to create a disjoint client training data and distribute each set of data to each client.

In an IID setting, we evenly divide the training dataset among all clients so that each client’s data distribution is identical and of the same size.

We assess the impact of different numbers of malicious clients under untargeted attacks, specifically Fang-Krum and Fang-Med. In this set of experiments, we fix the total number of clients and participation fraction at 10 and 1, respectively. To adhere to the assumption that the number of attackers is less than half of the participating clients, we investigate the test accuracy considering scenarios involving up to four malicious clients.

Figure 5 illustrates the accuracy in the given experimental settings, and it is evident that FedCC outperforms the other defense methods. We observe a general trend where the accuracy of defense methods decreases as the number of attackers increases. Notably, Coomed experiences the most significant drop in accuracy when transitioning from one to four malicious clients. This can be attributed to the fact that as the participation of malicious clients increases, there is a higher likelihood that the median values will be influenced by the malicious contributions.

In contrast, FedCC does not rely solely on geometric measures such as angles or distances but instead leverages hidden correlations between networks to identify and filter out malicious clients. As a result, FedCC demonstrates superior accuracy in filtering out malicious clients, regardless of the number of attackers involved. It is important to note that the accuracy degradation observed as the number of malicious clients increases primarily due to aggregating fewer clients rather than the malicious clients themselves being selected. These findings underscore the robustness and effectiveness of FedCC in defending against untargeted attacks, as it consistently outperforms other defenses across varying numbers of malicious clients.

We examine the impact of the participating fraction of clients where the number of total clients is fixed at 100. The fractions are 0.1, 0.3, and 0.5, such that the numbers of clients being aggregated are 10, 30, and 50, respectively. The proportion of malicious clients is fixed at 0.3, such that the malicious clients are 3, 9, and 15, respectively. Table IV summarizes the accuracy. We can observe that the accuracy tends to increase as the participation fraction is greater under Fang-Krum attacks. It implies that the more client participation in a non-IID setting, the more accurate the global model is if the attacks are mitigated. Under Fang-Med attacks, the accuracy is the highest with FedCC and second highest with Coomed. Throughout the experiment, FLARE prevented the global model from convergence, and the test accuracy fluctuated. The fluctuation is presumably due to the weight scaling while randomly selecting clients. Distinct from previous experiments, the server chooses a fraction of the client every round, meaning not-yet-trained models could be selected. In this environment, weight scaling leveraged by FLARE constantly improperly scales the local weights, causing divergence and fluctuation. These experimental results indicate that selectively choosing clients based on CKA similarity is more reliable than taking the median value of each coordinate in a non-IID setting. We also observed that as the participation fraction grows, the accuracy grows. This is due to the rising number of aggregated local models; the more models are aggregated, the more general global model is generated.

To examine the impact of the number of local epochs, we doubled up the number of local epochs such that the local models are trained for six epochs in benign clients and twelve epochs in malicious clients. The main task accuracy and confidence are summarized and illustrated in Table V and Figure 6. As expected, increasing the number of local epochs would not affect the accuracy or confidence but bring the convergence forward while being better at maintaining test accuracy. It is true that Coomed and FLARE significantly reduce the backdoor confidence, as well as FedCC; however, the test accuracy has even increased with FedCC due to its precise discrimination against malicious clients.

Figure 7 illustrates the distance between two clusters of clients using dendrograms with a single linkage metric and correlation distance method. The first two and last columns correspond to untargeted (Fang-Krum, Fang-Med) and targeted attacks, respectively. The total number of clients is set to 10; the participation fraction is 1.0. The number of malicious clients is 3 for untargeted and 1 for a targeted attack, with fedAvg. Observably, the height difference between the root of clusters is maximum in the second last layer (PLR).

The tables below summarize model architectures for each benchmark dataset (fMNIST, CIFAR10, CIFAR100). Considering that the end devices are generally incapable of handling heavy computation due to resource or communication constraints, we used lightweight models for the experiments.