Distributed Traffic Synthesis and Classification in Edge Networks: A Federated Self-supervised Learning Approach

Deep neural network (DNNs), are one of the promising solutions for network traffic classification [17, 15, 27, 28, 29, 30, 31, 32, 33, 34]. For example, Wang et al. proposed a hybrid neural network that combines a recurrent neural network (RNN) and a CNN to learn dynamic features of mobile data for traffic classification[15]. Liang et al. introduced a deep reinforcement learning-based approach, called NeuroCuts, for packet classification[30]. Zhang et al. proposed a deep learning-based traffic clustering solution that can classify packets associated with unknown classes and automatically build a new training dataset to update the classifier of unknown traffic[17]. Liu et al. applied an RNN-based flow sequence network approach to classify encrypted traffic flows[28]. Ensemble learning-based methods have been adopted to improve the traffic classification accuracy by Aouedi et al.[35] and Jin et al.[36].

FL and its applications in network data analysis have recently attracted significant interest [37, 38, 39, 40, 41, 42, 43, 44]. FL has the potential to protect data privacy during distributed training and coordination across decentralized datasets. Thus, network Wen et al. proposed a two-step FL framework to achieve privacy preserving model training with high communication efficiency. Wang et al. proposed an empirically driven solution to optimize the selection of client devices that participate in model updating[39]. Recently, FL was extended to data analysis in resource-limited networking systems. For example, Wang et al. investigated the convergence of gradient descent-based FL solutions and optimized the tradeoff between local update and global model aggregation under a given resource constraint[45]. Luo et al. introduced a low-cost sampling-based algorithm to learn the convergence-related parameters and minimize the cost of learning time and energy[46]. Xiao et al. proposed a federated edge intelligence (FEI) framework for implementing FL in edge computing-assisted IoT networks with communication and computational resource constraints[47]. This framework has been further extended to real-time learning in edge intelligence networks[48], as well as semantic communications[10] and semantic-aware networking systems[11, 49, 50]. Zhang et al.[51] studied the homomorphic encryption-based FL for communication and computation cost of private medical data analysis and Lu et al.[52] investigated FL-based imbalanced classification for industrial data.

Recently, self-supervised learning was introduced as a way to improve data efficiency and model generalization [20, 53, 54, 55, 19, 18, 56]. In particular, Araslanov et al. proposed a domain adaptation approach for semantic segmentation based on predictions produced by pseudo-supervision targets [20]. Li et al. proposed a two-stage framework for anomaly detection in which a self-supervised deep representation model was learned to build a generative classifier [53]. Sun et al. introduced a novel model training algorithm for Graph Convolutional Networks (GCNs). Self-supervised model in [54] improved model generalization performance while reduce the number of required labeled samples. Zhang et al. proposed a self-supervised learning method with adaptive memory network to improve model generalization and enrich feature representativeness[57]. Shi et al. studied a novel mask image model for self-supervised learning, where the idea of adversarial training was utilized[58]

Our proposed FS-GAN differs from traditional clustering-based data analysis techniques in that it adopts a self-supervised learning-based solution to first create synthetic samples with pseudo-labels that statistically match a mixture of real traffic types and then train an ML model using the pseudo-labeled datasets to classify and identify each individual service traffic.

The main architecture, including the key components and algorithmic procedures of FS-GAN, is presented in Fig. 2. FS-GAN consists of: (1) multiple generators that create synthetic samples whose statistical distribution is similar to that of real traffic in local datasets, and (2) multiple discriminators, each of which discriminator tries to differentiate the real traffic from the synthetic samples produced by the generators. Our proposed FS-GAN framework employs the following three procedures:

C-II requires less model-related data to be exchanged between edge servers and the coordinator, resulting in a higher communication efficiency. However, as will be shown later in this paper, C-I has faster convergence than C-II, which in some cases compensates for the extra communication overhead.

In the local data synthesis procedure, each discriminator only determines whether the data samples belong to a real traffic or are synthetic data produced by generators. After the global model coordination, all the discriminators should be able to differentiate real data in any local dataset from synthetic data generated by any generator. Similarly, the generators will be able to capture richer features across different local datasets. To address the model collapsing problem, we introduce a classifier in each local datatset to classify samples from different generators. As shown in Section 4, this classifier increases the divergence of the synthetic data distributions of different generators, hence alleviating the model collapsing problem of traditional GANs.

We present a theoretical analysis that proves the effectiveness of the FS-GAN framework. In particular, we prove that FS-GAN possesses two important properties: (1) the difference between the distributions of the mixture of the synthetic data generated by generators and that of the real traffic data distribution is minimized, and (2) the JSD divergence of the distributions of synthetic data samples generated by different generators is maximized. Based on these properties, we prove the main result of this paper, namely, the synthetic data samples generated by each individual generator captures the real traffic distribution of each individual traffic type, and that the classifier can differentiate all the synthetic samples coming from different generators, if the distributions of the traffic data associated with different services are separable.

Compared to existing self-supervised learning solutions, FS-GAN provides the following unique advantages that make it more suitable for network traffic classification problems. First, FS-GAN can be applied to a large network with highly decentralized and imbalanced traffic distribution with data privacy protection requirements. Second, FS-GAN does not require any pre-labeling of data, which makes it suitable for many of today’s networking systems that involve a large volume of unlabeled network traffic datasets that are prohibitively expensive to label. Third, the data synthesizing capability of FS-GAN makes it possible to be applied in many emerging networking services that involve traffic prediction and synthesis.

Compared to traditional traffic classification solutions, FS-GAN adopts a fundamentally different approach by first training multiple generator networks to generate synthetic samples that capture the distribution of real traffics associated with an individual service and then applying the parameters of the already trained models for classifying traffics of different services. This uniqueness of FS-GAN makes it applicable to a range of new application scenarios and use cases, including the following:

1) Dynamic Network Slicing in 5G/B5G: 5G NR release 16 [66] introduced the concept of network slicing, which focuses on isolating and preserving partitions of network resources and functions, commonly referred to as slices. These slices are tailored and orchestrated to support applications with diverse QoS requirements. One of the key challenges in network slicing is now to keep track of the traffic demands of potentially unknown applications, so the appropriate amount of resources can be reserved while meeting the needs of these applications. FS-GAN can be directly applied to classify and keep track of the needs of different services from a highly mixed traffic. It can also assist the network slicing manager in identifying newly observed service traffic types that do not have a sufficient amount of pre-labeled data.

2) Unknown Attack Detection: An important application scenario for traffic classification is attack detection. Existing solutions mostly focus on detecting known attacks. Detecting unknown attacks is a notoriously difficult problem[67]. FS-GAN has the potential to identify unknown attacks and simulate various attack scenarios as well as their mixtures.

3) Traffic Prediction-based Network Planning: Because the final model obtained by FS-GAN is capable of generating synthetic data samples that are statistically similar to the real data of various types of emerging traffic, FS-GAN can be applied to predict future traffic trends and assist in planning the network infrastructure to better cope with anticipated needs.

First, we consider local-model training of a single generator at the $d$th local dataset, $d=1,...,N$. As mentioned before, the main objective of this procedure is to train a model to generate synthetic data samples that captures the distribution of real traffic of a local dataset. Let $G_{d}$ and discriminator $D_{d}$ be, respectively, the generator and discriminator associated with the $d$th dataset. Training GANs can be formulated as a minimax game between $G_{d}$ and $D_{d}$ [59]. The generator and the discriminator are often DNNs. Let $P_{data}$ be the distribution of the real dataset and suppose that $\boldsymbol{x}$ is drawn from $P_{data}$, i.e., we write as $\boldsymbol{x}\sim P_{data}$. The generator $G_{d}$ aims to train a multi-layer neural network to map its input variables. Initially, a random noise $\boldsymbol{z}$ is generated to synthesize data samples whose distribution is denoted as $P_{\boldsymbol{z}}$. Without loss of generality, we abuse the notation and use $G_{d}(\boldsymbol{z})$ to denote the trained generator’s output given training input $\boldsymbol{z}$. Meanwhile, the discriminator $D_{d}$ tries to distinguish real data samples from synthetic samples produced by the generator. We use $D_{d}(\cdot)$ to denote the output of discriminator, which represents the probability that the discriminator decides the given input sample comes from the real dataset rather than being synthesized by the generator. More formally, we can express the adversarial training process between a single generator and a single discriminator via the following minimax problem:

The key idea of FS-GAN is to use multiple generators ${\boldsymbol{G}}_{d}=\{G_{d}^{1},G_{d}^{2},...,G_{d}^{M}\}$, rather than a single one to jointly train a model that can capture the distribution of a mixture of different traffic types in a given local dataset $d$. To address the issue of model collapse, we introduce a classifier network $C_{d}$ that classifies synthetic data produced by the generators. More specifically, the classifier first generates a pseudo-label for each synthetic data sample, indicating which generator it came from. It then minimizes its loss to encourage generators to create samples that are different from each other. As we prove later in this section, by employing $C_{d}$, each of the generators will synthesize data samples that represent a specific type of traffic, provided that the differences between the distributions of different traffic types are sufficiently large.

Now we focus on the training process for the multi-generator local model. We use the subscript to denote the index of the local dataset and the superscript for the generator index. Suppose a set ${\boldsymbol{G}}_{d}=\{G_{d}^{1},G_{d}^{2},...,G_{d}^{M}\}$ of $M$ generators has already been assigned to the discriminator $D_{d}$, where $d\in\{1,2,...,N\}$ represents the discriminator index. Each discriminator $D_{d}$ also interacts with a local dataset $X_{d}$, and each generator $G^{m}_{d},m\in\{1,2,...,M\}$, maps the input noise $\boldsymbol{z}$ to $\hat{\boldsymbol{x}}=G^{m}_{d}(\boldsymbol{z})$. The classifier $C_{d}$, is co-trained with ${\boldsymbol{G}}_{d}$ and $D_{d}$. Let $P_{G^{m}_{d}}$ be the distribution of the synthetic samples produced by $G^{m}_{d}$. We now describe the rule for each generator to interact with the discriminator. We set an index vector $\boldsymbol{v}$, randomly drawn from a predefined multinomial distribution $\boldsymbol{v}\sim{\cal M}\left(\boldsymbol{\pi}\right)$ where $\boldsymbol{\pi}=\langle{\pi}_{\boldsymbol{v}}\rangle_{\boldsymbol{v}\in{\boldsymbol{G}}_{d}}$ is the weighting coefficient of the distribution mixture. By setting $G^{\boldsymbol{v}}_{d}\triangleq\{G^{i}_{d}:i\in\ \boldsymbol{v}\}$ as the subset of ${\boldsymbol{G}}_{d}$, we can convert the multi-generator GANs into a standard GAN with a generator vector ${G}^{\boldsymbol{v}}_{d}$ and discriminator $D_{d}$. We use $P_{data}^{d}$ to denote the real distribution of dataset $X_{d}$. We can then formulate the model of local data synthesis with a multi-generator as the following minimax game:

where $C^{m}_{d}(\hat{\boldsymbol{x}})$ is the output of classifier $C_{d}$, specifying the probability that data sample $\hat{\boldsymbol{x}}$ is generated by $G^{m}_{d}$. The last term in (2) is a standard softmax loss in a multi-classification setting [68]. In other words, minimizing $C_{d}$ according to (2) under fixed $D_{d}$ minimizes the entropy for $C_{d}$, which leads to increased divergence among generators. We introduce a diversity hyper-parameter $\lambda>0$ to specify the degree that the classifier can influence the generators. We compare the model performance under different $\lambda$ in Section 5.

Let $\omega_{d}$ be the model parameters of the generators ${\boldsymbol{G}}_{d}$ associated with discriminator $D_{d}$, and let $\omega_{d}^{m}$ be the model parameters of the generator ${G^{m}_{d}},m\in\{1,...,M\}$. Classifier $C_{d}$ and discriminator $D_{d}$ share the same model parameters except for the last layer. Let $\theta_{d}$ be the model parameters shared between the classifier and the discriminator $D_{d}$. Note that the last layer of $C_{d}$ outputs an automatically learned pseudo-label for each input data sample, while the last layer of $D_{d}$ outputs a binary result. Similar to a standard GAN, we alternatively learn $\omega_{d}$ and $\theta_{d}$ using the stochastic gradient descend (SGD) method. The local data synthesis framework is illustrated in Fig. 3.

Let $B$ be the selected mini-batch size during the training process. At the beginning of each local training iteration, each discriminator $D_{d}$ will sample a mini-batch of $B$ data points $(\boldsymbol{x}_{d}^{(1)},\boldsymbol{x}_{d}^{(2)},...,\boldsymbol{x}_{d}^{(B)})$ from the real data distribution $P_{data}^{d}$. A mini-batch of $B$ synthetic data samples $(\hat{\boldsymbol{x}}_{d}^{(1)},\hat{\boldsymbol{x}}_{d}^{(2)},...,\hat{\boldsymbol{x}}_{d}^{(B)})$ will also be created by a mixture of generators $G_{d}^{v}$ in ${\boldsymbol{G}}_{d}$. This mixture can be generated according to a predefined sampling probability $\pi_{m}$. During each training iteration, we update $C_{d}$, $D_{d}$, and ${{G}^{\boldsymbol{v}}_{d}}$ along the gradients of their loss functions. According to (2), we can write the loss functions under the aforementioned settings as follows:

We summarize the detailed procedure in Algorithm 1.

In the previous section, the generators associated with a local discriminator produce synthetic data samples that capture the distributions of different traffic types in the local dataset. However, different local datasets may have different mixtures of traffic. It is therefore desirable to train a global model that can capture the diverse traffic across all local datasets. We adopt the FL-based framework to coordinate the local model training procedures without requiring any exchange of local datasets. One of the key ideas in FL is to coordinate different local model training procures via their model parameters. In one of the popular FL solutions, called FedAvg, each edge server periodically uploads its locally trained models to a global coordinator. The uploaded models are then averaged to form an updated global model, which will be broadcasted to all edge servers.

Formally, let $\Omega^{\prime}$ be the subset of discriminators in coordination scheme C-II (or discriminator-and-generator pairs in scheme C-I) that has been selected for uploading the local models. Note that $\Omega^{\prime}\subseteq\Omega$. Let $\omega_{0}$ and $\theta_{0}$ be the corresponding global model parameters, and let $p_{d}$ be the weight of the $d$th local dataset such that $p_{d}\geq 0$ and $\sum_{d=1}^{N}p_{d}=1$. We can write the global model coordination procedure as follows:

As mentioned in the previous procedures, the classifier shares the same network parameters with the local discriminator. Therefore, this classifier can be updated in the global model coordination procedure to classify and create pseudo-labels for the synthetic samples according to their corresponding generators. This classifier will also be able to classify the real data associated with all traffic types in all datasets. Since this classifier is a part of the training process of FS-GAN and is trained based on the pseudo-labeled synthetic datasets, it does not introduce any extra training efforts.

Since FS-GAN reuses the model parameters of discriminators and therefore does not have to introduce any new computational complexity, compared to the traditional federated learning implementation of GAN. In other words, the complexity of implementing FS-GAN is closely related to two types of implementation cost: communication and computational cost. The computational complexity of FS-GAN mainly includes the training of a set of local generative and discriminative models in parallel based on SGDs, i.e., the total number of local SGD rounds performed by edge servers are given by $O(MKT)$. The communication cost depends on the model size and the number of coordination rounds. In other words, FS-GAN introduces novel capabilities and address the issues of traditional federated GAN without introducing any extra computational and communication complexity.

We analyze FS-GAN from the following two aspects: data synthesis ability and classification accuracy across local models. Consistent with our previous notation, we use $P_{data}^{d}$ to denote the mixture distribution of real data samples in dataset $X_{d}$, and use $P_{model}^{d}$ to denote the distribution of the combination of synthetic data samples generated by generators $G^{1}_{d},G^{2}_{d},...,G^{M}_{d}$. First, we provide the optimal solution for classifier $C_{d}$ in Equation (2).

The result of the output of the optimal classifier ${C_{d}}^{*}(\boldsymbol{x};\theta_{d})$ can be seen as a weighted normalization of different synthesized sample distributions.

We follow a commonly adopted setting and use Jensen Shannon Divergence (JSD) [69] to quantify the difference between two distributions. For the optimal generator ${{\boldsymbol{G}}_{d}}^{*}(\boldsymbol{x};\omega_{d})$, we have the following proposition:

It can be observed that the two terms on the right-hand-side of (2) correspond, respectively, to the difference between distributions $P_{data}^{d}$ and $P_{model}^{d}$, and the inverse of the difference among the synthetic data generated by different generators. In other words, minimizing these two terms directly results in minimizing the difference between $P_{data}^{d}$ and $P_{model}^{d}$ while maximizing the differences among the generators.

Before presenting the main theorem, we make the following assumption:

Equation (9) means that the data distribution of data samples is a mixture of $M$ separable distributions given by $p_{d}^{m}(x)$ for $m=1,2,...,M$. We can prove the following theorem:

We can observe from (10a) that the classifier can correctly differentiate synthetic samples produced by different generators, i.e., the probability for the classifier to identify the correct generator $G_{d}^{m}$ that produces each given synthetic samples $x$ approaches one. As shown in (10b) and (10c), synthetic samples produced by optimal generator ${G_{d}^{m}}^{*}$ follow the same distribution as $p_{d}^{m}$. Also, all the generators weighted by $\pi_{m}$ together synthesize samples that match the same data distribution of that of the real dataset.

In cases where Assumption 1 is not satisfied, Theorem 1 can be considered as an upper-bound for the local data synthesis and classification performance. Existing works as well as our own experiments show that in many practical scenarios, even if Assumption 1 does not hold, FS-GAN can still create high-quality synthesized data with high classification accuracy performance. We will come back to this issue in Section 5.

So far, we have analyzed the optimal solution of the local model. Next, we focus on the convergence of the global model. In the global model coordination procedure, we use Federated Averaging (FedAvg) [65], a widely popular algorithm in federated learning, to periodically aggregate local models. More specifically, suppose that each participating local model performs $I$ local updates after receiving the latest global model. Let $T$ be the total number of local iterations performed by each client. Let $B$ be the mini-batch size. Let $\mathcal{L}_{d}(\boldsymbol{\varepsilon},\xi_{t}^{d}),d\in\{1,2,...,N\}$, be the loss function of local model in $t$th iteration with network parameters $\boldsymbol{\varepsilon}$ and a mini-batch of $B$ samples $\xi_{t}^{d}$. We define $\mathcal{L}_{d}(\boldsymbol{\varepsilon})$ as the mean value of $\mathcal{L}_{d}(\boldsymbol{\varepsilon},\xi_{t}^{d}),t=1,...,T$, i.e., we have $\mathcal{L}_{d}(\boldsymbol{\varepsilon})=\mathbb{E}[\mathcal{L}_{d}(\boldsymbol{\varepsilon},\xi_{t}^{d})]$. Consider the following optimization model:

Theoretical guarantees of (11) can be obtained by following the same approach as [70, 71, 43].

We can observe that Theorem 1 has proved that the proposed FS-GAN has addressed the two fundamental issues of federated learning and self-supervised GAN. In particular, compared to the federated learning which requires all the decentralized datasets to share the same combinations of data features, FS-GAN allows edge servers with different combinations of service traffics to jointly construct a shared model that can identify and synthesize service traffics of all the edge servers. Also, FS-GAN addresses the model collapse problem of GAN by reuse the model trained by the discriminator. Furthermore, as described in Section 4.4, FS-GAN does not introduce any new components or increase the complexity, compared to traditional federated GAN. This further justifies the flexibility and practicality of FS-GAN in practical networking systems.

To evaluate the performance of FS-GAN, we consider real-world dataset, “VPN-nonVPN dataset” (ISCXVPN2016)[26], to evaluate a highly mixed traffic data flow consisting of multiple types of unknown services. Within this dataset, we select data samples associated with 10 popular services, summarized in Table II. As our focus is on classification of valid information, we remove the PHY and MAC headers from the packets and limite the length of each payload to the same size of 2500 bytes so as to achieve a proper trade-off between the training efficiency and classification accuracy.

We conduct our experiments on a workstation with an Intel(R) Core(TM) i9-9900K CPU$@$3.60GHz, 64.0 GB RAM$@$2133 MHz, 2 TB HD and two NVIDIA Corporation GP102 [TITAN X] GPUs. Each edge server ia simuylated with a TITAN X GPU running on Ubuntu 16.04, Python 3.6, CUDA 10.0 and PyTorch 1.3.1. The training process between edge servers is simulated using the PySyft framework [72]. We combine the packet of all selected traffic types into one trace and equally divide the data into six local datasets, each being assigned to a discriminator (an edge server). The network architecture and parameter setting for the generator, discriminator, and classifier are summarized in Table III.

As mention earlier, compared to existing clustering solutions, FS-GAN introduces a novel clustering solution for model training by autonomously creating synthetic data with pseudo-labels. In this subsection, we compare the clustering performance of FS-GAN with existing clustering solutions. We consider three commonly adopted performance metrics:

Note that each of the above three metrics takes values between $0$ and $1$. The larger the better is the clustering performance.

In Table IV, we compare FS-GAN with four clustering solutions: K-means++, DEC[73], IDEC[74], and DCN[75]. It can be observed that FS-GAN achieves the best performance of all tested solutions in all three performance metrics. In particular, DEC, IDEC, and DCN use deep neural networks to recover different latent representations for clustering. Compared to K-means++, which directly searches for cluster centers or centroids of the data samples, deep-learning-based solutions often achieve better performance. FS-GAN takes a novel approach by first generating synthetic data samples with pseudo-labels and then training a classifier based on the labeled dataset. In some senses, this approach addresses the poor performance caused by the lack of labelled dataset and can result in improved performance when the quality of the pseudo-labeled synthetic data samples is high.

To compare FS-GAN with the most state-of-the-art self-supervised learning solution, we consider a recent extension of the DEC algorithm, called semi-supervised DEC (SDEC)[76], which incorporates the semi-supervised information learned from the labelled data samples in DEC to further improve the clustering performance. We use the SDEC as the pretext tasks and then include the peudo-labelled data samples into the self-supervised GAN algorithm to train the model, we referred to this approach as the SD-GAN. Note that, compared to FS-GAN, SD-GAN requires labelled datasets to learn a prior information to guide the learning process, it also requires an extra neural network model to perform the data clustering and peudo-labelling. In Fig. 6, we compare FS-GAN with SD-GAN with different setups under different number of training rounds, we can observe that FS-GAN-I outperforms SD-GAN and FS-GAN-II achieves similar performance in terms of NMI.

To compare the impact of model training parameters on the computational loads of FS-GAN under different setups, we present in Table V the running time per round of computation for FS-GAN in a fixed hardware and software environment under various performance metrics for 50 rounds of model training. We mainly evaluate the impact of four key parameters, including the number of local datasets, size of each dataset, the number of local training steps ($N$, $n$, $E$) between two consequent global model coordination, and the mini-batch size $B$. The learning results under one centralized dataset is also presented. We can see that both the learning time and performance of FS-GAN are slightly degraded compared with a centralized setting, because the model aggregation procedure is no longer needed. It can be observed that the computational load is most sensitive to the size of the dataset. Compared to scheme C-I, Scheme C-II always results in slightly more computational load for training. This is because, in scheme C-I, both generators and discriminators are coordinated, which accelerates convergence of the model training and leads to reduced overall computational time. The improved performance offered by Scheme C-I over C-II can be further shown in the NMI and ACC metrics. More specifically, C-I always exhibits a more accurate clustering performance than C-II when the number of training rounds is fixed. C-I and C-II offer similar performance in terms of RI. This is because RI measures the combined performance of the correct solutions over all the clustering results, instead of the highest performance among individual classes as NMI and ACC. Therefore, the performance difference in RI achieved by different schemes is always smaller than that of the other two performance metrics.

In Fig. 4, we present the clustering results when FS-GAN is used to classify the 10 services in Table II. Once again, FS-GAN delivers superior performance to IDEC and K-means++. Note that since K-means++ directly separates data samples based on the centroid, it tends to cluster most of the samples into a single or a limited number of clusters.

In Fig. 5, we compare the convergence performance based on NMI under different model parameters. We can observe that C-I always offers better convergence performance than C-II. This result is consistent with the previous observation. When the size of local dataset is the same, the convergence performance is closely correlated with this size. Specifically, the larger the dataset size, the faster is the convergence of the model training process. Note that the fluctuations in NMI is due to the interaction between generators and discriminators in the local data synthesis as well as the global model coordination.

As mentioned earlier, the classifier helps increasing the diversity of the synthesized samples produced by generators. At the same time, it interferes with the adversarial training process between the generator and discriminator. We investigate the impact of the classifier on the data clustering performance by adjusting the diversity hyper-parameter $\lambda$ in Equation (2). Intuitively, when the penalty brought by the classifier becomes larger than that of the discriminator, generators will be encouraged to produce more diversified samples even if these samples may not follow the same distribution as the real data. In Table VI, we present the clustering performance of FS-GAN for different values of $\lambda$. We can observe that a very large or small value of $\lambda$ will not typically offer the best performance The optimal $\lambda$ varies with the model setup. Finding the optimal $\lambda$ that achieves the right balance between classification and discrimination will be left for future work.

As mentioned earlier, FS-GAN is more than just a traffic classification approach. It can also learn from the decentralized datasets and produce synthetic data samples that capture the distribution of real data associated with unknown services. In this subsection, we evaluate this aspect of FS-GAN.

We compare the data synthesis capability of FS-GAN to various extensions of multi-generator GANs with FL, which we refer to as F-GAN. In particular, we compare the following 4 different schemes:

• 1)

  F-GAN (three-G): 3 generators for each dataset without the classifier.

• 2)

  FS-GAN (three-G): 3 generators for each dataset.

• 3)

  F-GAN (six-G): 6 generators for each dataset without the classifier.

• 4)

  FS-GAN (six-G): 6 generators for each dataset.

As mentioned earlier, one of the key challenges of GAN-based solutions is the possibility of triggering a model collapse problem where, in this case, different generators will only produce samples with limited variety regardless of the input. In Fig. 7, we evaluate the diversity of synthesized data samples produced by different schemes. We compare the default numbers of synthetic samples produced by each scheme that are associated with different services when the same input is employed. It can be observed that different generators tend to produce different numbers of synthetic samples. For example, among all six types of services, SCP (download) is the least popular service to be synthesized with the least number of synthesized data samples, e.g., less than 0.02% of total synthesized samples produced by F-GAN(C-I, six-G) falls into this service category. We, however, observe that FS-GAN, especially C-I scheme with six generators, produces relatively uniform number of samples for different services. This means that FS-GAN offers the best performance in terms of balancing synthesized data samples.

To quantify the diversity of the synthesized samples, we use statistical distance metrics, which measure the difference between the distribution of the synthetic data samples and that of the real service traffic data. We consider three distance metrics: Kullback-Leibler (KL) divergence, Jensen Shannon (JS) divergence [69], and Wasserstein (W) distance [77], consider two random variables, $p$ and $q$, with respective distributions $p(x)$ and $q(x)$. Table VII provides the distance results under both FS-GAN and F-GAN. FS-GAN achieves the smallest distance between synthetic and real data, where the JS divergence under FS-GAN-I is shown to be as low as 0.011, almost one fifth of the lowest distance result achieved by F-GAN. This means that the proposed FS-GAN is ideal to classify and synthesize highly heterogeneous traffic flows with mixed services.