Examining Machine Learning for 5G and Beyond through an Adversarial Lens

A schematic diagram of the 5G network architecture is depicted in Figure 1. Apart from the user equipment (UE), the 5G system features a cloud-native core network, a flexible and disaggregated radio access network (RAN), and a provision for multi-access edge (MEC) cloud for reduced latency. The gNodeB (gNB) of the RAN comprises of split-able access nodes, distributed and centralized units (DU and CU), to efficiently handle evolved network requirements. The gNB connects to the MEC to significantly reduce the network latency for selected applications by availing edge server computing at the MEC cloud which is close to the radio service cells. For instance, to cater to the ultra-reliable low-latency communication (URLLC) use-case of industry automation, the RAN radio unit along with the DU, CU, and the MEC can be installed onsite. Thus, 5G network architecture enables applications to be deployed remotely (App 3 and App 4) or near the edge (App 1 and App 2) where low latency is a requirement. The provision of MEC also reduces the aggregated traffic load on the transport networks that is responsible for connecting RAN to the core network. The 5G core network (5G-CN) is a cloud-native network that stores subscriber databases and hosts essential virtualized network functions for network operations and management. Although, the network management and control functions are shown to be co-located with the core in the figure, they can be flexibly deployed at the edge as needed.

A wide spectrum of DL algorithms are being developed for the broad context of wireless communications and 5G networking to deal with problems that are either hard to solve or hard to model [1]. For instance, optimal physical network resource allocation for NFV is an NP-hard problem and so require exponential computational power with increasing system size [2]. Deep RL (DRL)-based solutions are proposed to efficiently address resource allocation problems (e.g., [3]). Network channel estimation for efficient beamforming is a problem that is hard to model and deep neural network (DNN)-based SL solution is well-accepted to tackle it [4]. Moreover, in certain use-cases, conventional expert systems become inappropriate due to real-world constraints, such as limited availability of power, where AI can perform effectively. For instance, deep autoencoder based systems can replace the power-hungry RF chain hardware with small embedded sensor systems enabling them to sustain longer on onboard power supplies [5]. DL algorithms generally outperform the conventional approaches in solving mobile network prediction problems such as physical layer channel prediction by SL, signal detection problems such as recovering transmitted signals from noisy received signals by UL, and optimization problems like resource allocation by RL.

The vulnerability of the ML algorithms, especially the DL models, to the adversarial attacks is now well-established, where adversarial inputs are small carefully-crafted perturbations in the test data built for fooling the underlying ML model into making wrong decisions. An adversary can often successfully target an ML model with no knowledge of the model (black-box attack), or some knowledge (grey-box attack), or full knowledge (white-box attack) of the target model. An adversary can attack the model during its training phase and in its testing phase as well. The training phase attacks are known as “poisoning attacks" and the test time attacks are known as “evasion attacks". Evasion attacks are commonly known as adversarial attacks in the literature [9].

More formally, an adversarial example $x^{*}$ is crafted by adding a small indistinguishable perturbation $\delta$ to the test example $x$ of a trained ML classifier $f(.)$ where $\delta$ is approximated by the nonlinear optimization problem provided in equation 1, where $t$ is the class label.

In 2013, Szegedy et al. [10] observed the discontinuity in the DNN’s input-output mapping and reported that DNN is not resilient to the small changes in the input. Following on this discontinuity Goodfellow et al. [11] propose a gradient-based optimization method for crafting adversarial examples. This technique is known as fast gradient sign method (FGSM). Papernot et al. [12] craft adversarial perturbation using a saliency map-based approach on the forward derivatives of DNN. This approach is known as Jacobian saliency map based attack (JSMA). Carlini et al. [13] crafted three different adversarial attacks using three different distance matrices ($L_{1}$, $L_{2}$, and $L_{\infty}$). More details about adversarial ML attacks are described in [9, 7].

5G and B5G networks embrace ML-driven solutions for improved accuracy and smarter network operations at scale. Figure 2 illustrates network problems from different network segments of 5G, namely user devices, RAN, MEC, core networks, and the network management and control layer that have recently attracted ML-based solutions from all the three categories of ML. However, in light of the above discussion in §III-A, the DL-powered ML models gaining popularity for 5G and B5G networks are vulnerable to the adversarial attacks thereby further aggravating the security risks of future generations of mobile networks.

In our attempt to show the feasibility of adversarial ML attacks on 5G systems we take three well-known ML models—one from each of the three ML families of algorithms (UL, SL, and DRL)—from wireless physical layer operations relevant to 5G and B5G context and show the vulnerability that naive use of ML brings to future mobile networks. We choose all the three ML models for our case studies from the physical layer network operations because of the maturity of ML-research in the context of AI-driven 5G networking and the availability of open-sourced ML models backed up with accessible data-sets¹¹1https://mlc.committees.comsoc.org/research-library/.

Automatic modulation classification is a critical task for intelligent radio receivers where the signal amplitude, carrier frequency, phase offsets, and distribution of noise power are unknown variables to the receivers subjected to real-world frequency-selective time-varying channels perturbed by multipath fading and shadowing. The conventional maximum-likelihood and feature-based solutions are often infeasible due to the high computational overhead and domain-expertize that is involved. To make modulation classifiers more common in modern 5G and B5G networked devices, current approaches deploy DL to build an end-to-end modulation classification systems capable of automatic extraction of signal features in the wild [14].

We pick a convolutional neural network (CNN)-driven SL-based modulation classification model in this case study to illustrate the added dimension of vulnerability introduced in the networks by it. We use the well-known GNU radio ML RML2016.10a dataset that consists of 220000 input examples of 11 digital and analog modulation schemes (AM-DSB, AM-SSB, WBFM, PAM4, BPSK, QPSK, 8PSK, QAM16, QAM64, CPFSK, and GFSK) on the signal to noise ratio (SNR) ranging from -20 dB to 18dB [15]. However, we exclude the analog modulation schemes from our study and consider only the eight digital modulations from the data set because from 2G onward all mobile wireless standards are strictly digital communications. Figure 3 depicts the classification performance of the CNN model in the multi-class modulation classification for the signals between -20dB to 18dB of SNR.

To show the feasibility of an adversarial ML attack on the CNN-based modulation classifier we make the following assumptions:

• •

  We consider the white-box attack model where we assume that the adversary has a complete knowledge about the deployed modulation classifier.

• •

  Goal of the adversary is to compromise the integrity of the CNN classifier leading to a significant decay in the classification accuracy which is the measure of the success of the adversary.

To craft the adversarial examples to fool the CNN classifier, we use the Carlini & Wagner (C&W) attack [13] for each modulation class by minimizing the $L{{}_{2}}$ norm on the perturbation $\delta$, such that when the perturbation $\delta$ is added to the input $x$ and sent to the CNN-based modulation classifier $C$ it misclassifies the input $x$. More details on the C&W attack are available in [13]. The performance of the CNN-based modulation classifier before and during the adversarial attack is depicted in Figure 3. A distinct drop in the accuracy of the modulation classification after the adversarial attacks indicates the brittleness of deep supervised ML in 5G and B5G applications. Moreover, our results show that the approbation of unsafe DL models in the physical layer operations of the 5G and B5G networks can make the air-interface of the future networks vulnerable to adversarial ML attacks.

In 2016, O’Shea et al. proposed the idea of channel autoencoders which is an abstraction of how an end-to-end radio communication module functions in real-world wireless systems [17]. The deep autoencoder-based communication model gains rapid popularity and is seen as a viable alternative to the dedicated radio hardware in the future 5G and beyond networks [5]. Figure 4(a) depicts the conceptual design of the channel autoencoder that we choose as a deep UL model for this case study. We assume the model is subjected to an additive white Gaussian noise (AWGN) channel and apply the parameter-configurations provided in [16]. To perform the adversarial ML attack on the channel autoencoder we consider the following threat model and compare the performance of the model with and without attack.

• •

  We assume a white-box setting, where the adversary has complete knowledge of the deployed ML model. We further assume that the autoencoder learns a broadcast channel. The proposed adversarial attack on channel autoencoder can be converted into a black-box adversarial attack, where the adversary has zero knowledge of the target ML model, by following the surrogate model approach provided in [18].

• •

  The goal of the adversary is to compromise the integrity of channel autoencoder and the success of the adversary is measured by the elevated block error rate (BLER) with improving SNR per bit ($E_{b}/N{{}_{0}}$).

We take the following two-step data-independent approach to craft adversarial examples for the channel autoencoder:

1. 1.

   Sample the Gaussian distribution randomly (we sampled Gaussian distribution because the channel is AWGN) and use it as an initial adversarial perturbation $\delta$;

2. 2.

   Maximize the mean activations of the decoder model when the input of the decoder is the perturbation $\delta$.

This produces maximal spurious activations at each decoder layer and results in the loss of the integrity of the channel autoencoder. Figure 4(b) shows the performance of the model before and under the adversarial attack. Moreover, the figure suggests the adversarial ML attack is often outperforms the traditional jamming attacks.

Since the idea of channel autoencoder in a wireless device is to model the on-board communication system as an end-to-end optimizable operation, the adversarial ML attacks on channel autoencoder show that the application of unsupervised ML in the 5G mobile networks increases its vulnerability to adversarial examples. Hence, we argue that deep UL-based 5G networked systems and applications need to be revisited for their robustness before being integrated into the 5G IoT, and related systems.

In the final case study, we have performed the adversarial ML attacks on an end-to-end DRL autoencoder with a noisy channel feedback system [19]. Before providing the details of the adversarial ML attack on the DRL-based communication system, we briefly discuss the DRL-based channel autoencoder. Goutay et al. [19] take the same architecture we consider in the previous case study §IV-B and add a noisy feedback mechanism to it, as shown in Figure 5(a). The end-to-end training procedure involves:

1. 1.

   The RL-based transmitter training by a policy gradient theorem [20] to ensure that the intelligent transmitter learns from the noisy feedback after a round of communication.

2. 2.

   SL model-based receiver training to train the receiver as a classifier.

Both transmitter (encoder) and receiver (decoder) are implemented as separate parametric functions (differentiable DNN architectures) that can be optimized jointly. The communication channel is modeled as a stochastic system that provides a conditionally distributed relation between the encoder and the decoder of the channel autoencoder. More details on the design and training procedure are available in [19]. The considered threat model for this case study is given as:

• •

  We choose a black-box settings where the adversary does not know the target model. We assume black-box settings because these are realistic as the attacker has no information about the deployed encoder and decoder model, and the adversary can only attack by adding perturbation in the broadcast channel. We also assume that the adversary can perform an adversarial ML attack for “$n$”-time steps.

• •

  The goal of the adversary is to compromise the performance of the DRL autoencoder with noisy feedback for a specific time interval. The success of the adversary is measured by the degradation in the decoder’s performance during the attack interval.

To evaluate the robustness of DRL model, we exploit the transferability property of the adversarial examples, which states that adversarial examples compromising an ML model will compromise other ML models with high probability if the underlying data distribution is same between these two models. We opt for the surrogate model approach for performing the adversarial ML attack. So we transfer the adversarial examples crafted in case study (§IV-B) and measure the average accuracy of the receiver. We run the DRL autoencoder with a noisy feedback system for 600-time steps (one time-step is equal to one communication round) and perform the adversarial attack between 200 to 400-time step window. We transfer 200 successful perturbations from the previous case study (§IV-B) and report the performance for the DRL system. Figure 5(b) provides the performance of the receiver (decoder) of the DRL autoencoder. It is evident that the performance of the receiver degrades from 95% to nearly 80% during the adversarial attack window. The recovery in the performance after the adversarial attack explicates that the end-to-end DRL autoencoder communication system can recover because of the noisy feedback and DRL adaptive learning behavior.

Our results, as presented in this section, confirm the feasibility of adversarial ML attacks on DL-based applications from all the three types of ML algorithms that are prevalent in the 5G network systems and highlight the additional threat landscape emerges due to the integration of vulnerable DL models to the 5G and B5G networks.

Robustness against adversarial ML attacks is a very challenging problem. To date, there does not exist a defense that ensures complete protection against adversarial ML attacks. In our previous works [9, 21], we have performed an extensive survey of the adversarial ML literature on robustness against adversarial examples, and showed that nearly all defensive measures proposed in the literature can be divided into:

1. 1.

   modifying data approaches (adversarial training, feature squeezing, input masking, etc.);

2. 2.

   auxiliary model addition approaches (generative model addition, ensemble defenses, etc.);

3. 3.

   modifying model approaches (defensive distillation, model masking, gradient regularization, etc.).

Our results in §IV indicate that ML-based 5G applications are very vulnerable to the adversarial ML attacks. There does not exist much work on the recommendations and guidelines for evaluating the robustness of ML in 5G applications. In the following, we have provided a few important evaluation guidelines for evaluating the ML-based 5G applications against adversarial ML attacks. These insights are extracted from the Carlini et al. [22] and our previous works [23, 9].

• •

  Many defenses are available in the literature against adversarial attacks but these defenses are limited by the design of the application. Using them without considering the threat model of ML-based 5G applications can create a false sense of security. So for ML-based 5G application threat models must clearly state the assumptions taken, type of the adversary, and the metrics used for evaluating the defense.

• •

  Always test the defense against the strongest known attack and use it as a baseline. Evaluating for an adaptive adversary is also necessary.

• •

  Evaluate the defense procedure for gradient-based, gradient-free, and random noise-based attacks ²²2https://www.robust-ml.org/.

• •

  Clearly state the evaluation parameters (accuracy, recall, precision, F1 score, ROC, etc.) used in validating the defense and always look for a change in the false positive and false negative scores.

• •

  Evaluation of the defense mechanism against out-of-distribution examples and transferability-based adversarial attacks is very important.

Although these recommendations and many others in [23, 9, 21, 22] can help in designing a suitable defense against adversarial examples but this is still an open research problem in adversarial ML and ripe for investigation for ML-based 5G applications.

Apart from the vulnerability of the ML models to the adversarial ML attacks, we underline the following drawbacks that call into question the viability of ML-driven solutions to be integrated into the real-world 5G networks any time soon.