Security Analysis of SplitFed Learning

Federated Learning is a decentralized machine learning mechanism where each client has the same copy of a neural network and its own training data. In each round of training, each client computes forward and backward passes over its model to calculate updated weights. It then sends the updated weights to a central server where all other clients pool their updated weights as well. The central server then performs an aggregation algorithm, such as FedAvg  (McMahan et al., 2017), and computes a global update. This global update is nothing but the average of all the clients’ individual weights. This global update is then shared with each of the clients who start the next round of training with the same weights. This is performed continuously until the model converges. Federated learning thus achieves model training without sharing local training data with other clients or the server involved in the training process.

People have exploited the security vulnerabilities of Federated Learning and shown that it is susceptible to model poisoning attacks that severely degrade the accuracy of the model  (Fang et al., 2020; Shejwalkar and Houmansadr, 2021). The LIE(Little is Enough) Attack  (Baruch et al., 2019) takes the mean $\mu$ and the standard deviation $\sigma$ of the updates and crafts a malicious update $k=\mu+z*\sigma$ where the coefficient $z$ controls the amount of deviation. Much stronger than these attacks is the dynamic optimization-based attack proposed by  (Shejwalkar and Houmansadr, 2021) and we use this attack in our study. This attack is described in detail in Section  3.2.

Split Learning  (Gupta and Raskar, 2018) is another distributed machine learning framework. It works by splitting the machine learning models between a client and a server. The model is split at some intermediate layer which we will refer to as the cut layer throughout this paper. The output of this cut layer is referred to as smashed data. The initial layers are in the client, while the rest of the layers are on the server. Therefore, the server performs most of the computation but since it has access to only the second part of the model, it does not have access to the data.

The client performs a forward pass over its portion of the model and sends the smashed data to the server. The server continues the forward propagation over its portion of the model, calculates loss, and begins back-propagation up to the cut layer. It then sends gradients, calculated up to the cut layer, to the client to complete back-propagation. This constitutes one training round of SL. It then sends the updated weights to the next client. The next client starts training from that updated model using its own dataset. The training process is sequential and circular meaning that once we reach the last client, the next client is the first client again.

SplitFed Learning is a new variant  (Thapa et al., 2022) of the distributed machine learning framework which combines both Split Learning and Federated Learning. Although Split Learning enhances model privacy by splitting up the model and the server does not know about the client’s side of the model or the data, it is sequential in nature and takes more time to train. Compared to Split Learning, Federated Learning has lower communication efficiency because Split Learning has lesser bandwidth requirements as we are not sharing the complete model at any point. SplitFed combines the parallel computation from Federated Learning and the resource-efficiency and improved model privacy from Split Learning. The working of the SplitFed model is shown in Figure  1. Similar to the classic Split or Federated Learning settings, we have a set of $N$ clients and $N$ models here. Each of the $N$ models is split at the cut layer between the client and the server. The system model is similar to the Split Learning model except that now we have a FedAvg server on the clients’ side, just like in the Federated Learning system model.

Each client computes its forward pass in parallel and sends the intermediate outputs from the $cut\ layer$ to the server. The server computes the forward and backward passes on each of the clients’ outputs and sends them back to the clients. The clients perform the backward pass on their models and send their parameters to the Fed Server. The Fed Server performs Federated Averaging and sends back the parameters to the client.

There is a variant of the SplitFed technique where the server-side computation is done in parallel on each of the client outputs. It completes in lesser time but has reduced accuracy overall. Similar to the client side, it requires an aggregation server on the server side which computes an aggregated server-side model at the end of each global epoch. However, we do not cover that variant in this work.

In our threat model, the adversary attempts to compromise model training by injecting malicious updates into the system. The adversary has access to the model on the compromised client. It can modify the weights and gradients of that model. The adversary controls M out of the total N clients. In our work, we set M/N to be 0.2 i.e., 20% of the clients are malicious, which is practical in current deployments. In addition to this, we assume an honest server. Our assumptions are similar to previous works that have done client-side attacks  (Shejwalkar and Houmansadr, 2021; Pasquini et al., 2021), where the server is kept honest (from service providers such as Google, Amazon) and provides legitimate service. We also assume that low dimensional model updates are possible. In a practical setting, this might be the case when multiple hospitals are collaborating to jointly train some cancer detection model and some of those are malicious and wish to corrupt the training process.

The focus of this work is on model poisoning attacks on compromised clients, and not on data inference attacks on compromised servers (Fang et al., 2020). We consider the case where the adversary has knowledge of the model gradients on benign clients, but it can only modify the model gradients on malicious clients. This scenario serves as the upper bound for the efficacy of our attacks and the robustness of our aggregation rules, trimmed mean and median, which we describe in detail in Section  3.2.

Here we describe the state-of-the-art model poisoning attack which we use in our work. In this type of attack by  (Shejwalkar and Houmansadr, 2021), we formulate an optimization problem given by equation 1 that crafts a malicious update which is then sent by the malicious clients to the Fed Server for aggregation. The attacker takes the average of the benign model updates (equation 2) and perturbs them with a value $\gamma*\nabla^{p}$ as shown in equation 3 and this is the highest value of $\gamma$that bypasses the aggregation rule being used. The optimization problem is formulated as follows:

(1)

$$\underset{\gamma}{\arg\max}=\nabla^{b}-f_{\text{agr}}(\nabla^{m}_{i\in[m]}\cup\nabla_{i\in[m+1,n]})$$

Here the average of benign gradients is given by:

(2)

$$\nabla^{b}=f_{avg}(\nabla_{{i\in[n]}})$$

and each malicious update is given by:

(3)

$$\nabla^{m}_{{i\in[m]}}=\nabla^{b}+\gamma*\nabla^{p}$$

There are different perturbation types, such as unit vector, standard deviation, and opposite direction. In our case, we will be using the standard deviation perturbation as it enables a stronger attack.

(4)

$$\nabla^{p}_{std}=-std(\nabla_{{i\in[n]}})$$

This attack is designed to break through state-of-the-art robust aggregation mechanisms such as trimmed mean and median. Next, we describe those aggregation mechanisms briefly and how the optimization framework is tailored to break through trimmed mean and median.

Trimmed Mean is an aggregation rule  (Yin et al., 2018),  (Xie et al., 2018) that replaces the simple averaging technique. It takes the set of all updates, including malicious ones, as input, sorts the updates along each dimension $i$, and removes $m$ of the largest and smallest values from the set. Note that, in SplitFed, these updates are from the portion of the model on the clients only. Here, $m$ is the number of malicious clients. Then it takes the mean of the updates along each trimmed dimension. The aggregated update is then sent back to each client. The general optimization framework given in equation  1 is modified for trimmed mean where $f_{agr}$ is now $f_{trmean}$ as done by  (Shejwalkar and Houmansadr, 2021). Here, the trimmed mean is computed on the aggregate of malicious and benign updates, and the adversary’s objective is to maximize the norm between this trimmed mean and the mean of benign updates.

Median is similar to trimmed mean  (Yin et al., 2018),  (Xie et al., 2018) in the sense that it also removes malicious update components along each dimension of the update by means of a statistical operation. It computes the element-wise median of the updates along each dimension. The general optimization framework given in equation 1 is modified in a similar fashion to the trimmed mean attack. The only difference it has is that $f_{agr}$ is now $f_{median}$.

Alexnet  (Krizhevsky et al., 2012) is a standard model with some convolutional layers followed by a fully connected layer. We train it on the CIFAR10 dataset for 1000 epochs with a learning rate of 0.01, batch size of 64, and an SGD optimizer. We split the model at three different layers. More specifically we split it at the first, second, and third maxpool layers. We call these split versions V1, V2, and V3 respectively.

To ensure the reproducibility of our results we fix the random seeds to be 42 for PyTorch, NumPy, and Python’s random package. This is because there are several parts of the code where randomness is taking place, such as the dataloader, weight initializations etc. Reproducibility is even more important in our non-IID case of the FEMNIST dataset where we want the same client selection sequence for each experiment trial we conduct.

We report the efficacy of the attack in terms of accuracy drop ($Acc_{drop}$) that quantifies the difference between the test accuracy obtained without attack ( $Acc$)and the test accuracy obtained with attack ($Acc\textsubscript{$attack$}$)

(5)

$$Acc\textsubscript{$drop$}=Acc-Acc\textsubscript{$attack$}$$

Simply reporting the accuracy after the attack would not be a fair comparison because we get different accuracies for different models and model splits.

To test the case of a practical setting, we evaluate our setup with 100 clients. With our setting of 20% malicious clients, 20 of the clients from among our 100 are malicious. We use several NVIDIA RTX8000 GPUS, each having 48GB memory on our university-provided GPU cluster to run our experiments. All the code was written in PyTorch(1.12.1+cu116) and run on Jupyter Notebooks on the GPU cluster. We perform hyperparameter tuning with lr=[0.01, 0.05, 0.1], momentum = [0.9, 0.95, 0.99], and batch_size = [32, 64, 128].

We have seen that the choice of different cut layers does not functionally change the split learning system. However, the choice of cut layer plays a huge role in the effectiveness of our attack. We see that when the client has a smaller portion of the full model the attack does not reduce the overall accuracy much. Conversely, when the client has a larger portion of the model, the adversary has a lot of space to manipulate the model. In this scenario, the overall accuracy of the final model is severely degraded.

As shown in Table 2, in the case of Alexnet trained in CIFAR10, when we increase the percentage of the model on the client side from VGG v1 to VGG v3, and the accuracy drop increases from $9.05\%$ to $41.42\%$ for trimmed mean. When we compare it with the federated learning case in the same setting and it gives a higher accuracy degradation of ($49.3\%$) as shown in Table 1. because it has the complete model on the client side. Figures  2 and  3 show this accuracy as the $FL\ baseline$ (shown as red dashed line). We also observe that the $v3$ model split has almost equal accuracy drop as the baseline (FL) however the accuracy drop of $v2$ split is lower than $v3$ but still significantly higher than that of $v1$. Based on these results, we can advocate the use of $v1$ versions of our models for more robustness against poisoning attacks. This can also prove to be beneficial for resource-constrained devices and could lead to faster training time.

Next, we compare the two defenses, the trimmed mean and median in our evaluation as shown in Tables 1 and Table  2. We can see throughout all settings with different model architectures, datasets, and model splits median defense performs better than the trimmed mean defense. The highest performance difference is for Alexnet (v3), where the median is about $11\%$ better than trimmed mean. For the Federated case, we see the same trend with median with $8-10\%$ better performance than the trimmed mean. This trend is consistent with FEMNIST too for both FL and SplitFed. From this we can conclude that the median defense is more robust than the trimmed mean defense. This may be due to reason that median is a much more robust measure of central tendency than mean in the presence of an outlier that can still exist when the upper and lower m values are trimmed. Therefore, it is much harder for the adversary to craft an update that bypasses the median defense.

Initially we perform the attack on non-IID FEMNIST with the same learning rate as in IID CIFAR10(lr=0.01). Similarly to our IID case, we do poisoning at every training round. This leads to very low model accuracy (below $10\%$/) because the adversary gets too much attack space in non-IID because the model updates are very different from each other, and the high learning rate allows it to craft a malicious update of large magnitude. This effect is also validated in  (Shejwalkar and Houmansadr, 2021; Fang et al., 2020). We then perform the attack with a lower learning rate of $0.001$ and carry out poisoning after several training rounds without poisoning. We report these results in Tables 1 and Table 2. A significant observation in this case is when we use the $v1$ model split, where we do not get a drop in accuracy with our attack. This is because the model we are using is already small in size, and the adversary gets an even smaller part to attack.

We need to determine what percentage of clients would be suitable to perform significant accuracy degradation. Although it is intuitive that to get the highest degradation, we need a higher percentage of malicious clients, we need to keep practical considerations in mind. It is not possible to have a very large number of malicious clients in a practical setting  (Shejwalkar et al., 2022). We tested our attack with a varying percentage of malicious clients for the CIFAR10 dataset. We see that the accuracy of the model starts to degrade with a tiny percentage of malicious clients (2%) and that the accuracy of the model shows a sharp degradation at 10% malicious clients. For the Alexnet V2 model, we see a complete degradation in model accuracy when we increase the percentage of malicious clients to 30%. In this scenario, the accuracy of the model is about 10% which is equivalent to a random guess for a classification problem of 10 classes, these trends are comparable with Federated Learning as well, as reported in  (Shejwalkar and Houmansadr, 2021). We observe that the same increase in the percentage of malicious clients in Federated Learning causes a much larger change in accuracy drop as compared to SplitFed. Increasing the percentage of malicious clients from $10\%$ to $15\%$ for CIFAR10 with Alexnet and trimmed mean increases the accuracy drop by $11\%$ in Federated Learning while the accuracy drop in SplitFed is $4\%$, as shown in Figure 4. We see this effect for every $5\%$ increment in malicious clients from $0\%$ to $30\%$. This observation is also consistent with the FEMNIST dataset.

So far we have performed the attack at the client-side model aggregator, to perform a comparison with Federated Learning. It would be interesting to see the effect of performing some perturbation on the smashed data sent by some malicious client to the server. The dimensionality of smashed data would change with different choices of the cut layer and this would most certainly affect the success of the attack.

We performed an extensive analysis of model poisoning on the SplitFed system by using existing state-of-the-art attacks. We test these attacks by pairing them with various defenses, models, datasets, data distributions, and model splits and find that the same attacks in SplitFed have lower efficacy than in Federated Learning because in SplitFed the client does not have the complete model. We also propose an additional attack space that can be explored in future work.