PANTS: Practical Adversarial Network Traffic Samples
against ML-powered Networking Classifiers

Consider a network operator that uses an ML-based traffic classifier to distinguish real-time applications (e.g., online gaming). The operator uses this information to optimize routing: real-time applications use fast but low-bandwidth network paths. This is one of the multiple network management tasks for which network operators could use a ML-based traffic classifier (built with Multilayer Perceptron-MLP, Random Forest, Transformers [25, 6, 18, 33, 11]) and benefit from its ability to learn complex patterns directly from data. While highly useful, this MNC exposes the network to various exploits. Two representative examples include attackers controlling (i) the end hosts of the communication or (ii) an in-network component. In the former case, a malicious (or simply “rational”) application developer could manipulate the traffic generated by her application to be misclassified as gaming by the network’s MNC, hence gaining an unfair performance advantage over competitors. This could be achieved by subtly altering the size of some packets or delaying specific packets without significantly impacting the overall performance of her application’s traffic. In the latter case, a malicious or compromised router in-path can increase the volume of traffic routed through the fast but low-bandwidth path by subtly altering the non-real-time traffic to cause the MNC to misclassify it as real-time. These perturbations would result in congestion in the low-bandwidth path, leading to delays or even downtime for actual real-time traffic, which is vital for the network’s profitability. The attacker cannot inject packets because that would be detectable from the application perspective, but she can delay packets.

Next, we elaborate on the requirements of a framework that generates adversarial samples and use them to enhance the robustness of MNCs before we explain why existing approaches fall short.

Having described the requirements of a framework that aims to help network operators, we explain why various lines of work that generate synthetic or adversarial networking inputs fail to satisfy them. We elaborate on works that generate adversarial images (i.e., AML) in §3.2.

Consider an ML model $f$ used to implement a network classifier $f:\mathcal{R}^{n}\rightarrow\mathcal{Y}$ where $\mathcal{Y}$ represents the set of possible labels. We define ${x}\in\mathcal{R}^{n}$ as the input to $f$. This can be the result of a feature engineering function, denoted as $\mathbf{\phi}$ on a sequence of packets p= $\{p_{1},p_{2},\dots,p_{k}\}$ from the set $\mathcal{P}$ where each $p_{i}$ is a packet arrived at time $i$ or a subset of the sequence of packets p. Each packet can be represented by various attributes, including its size flags, direction etc. We assume that for every MNC, the following dataset $D$ exists. Let $D=\{(x_{i},y_{i})\}_{i=1}^{m}$ be the dataset, where $x_{i}\in R^{n}$ are the feature vectors obtained by applying $\phi$ to raw network data and $y_{i}\in Y$ are the corresponding label.¹¹1Applying $f$ directly on a truncated packet sequence is possible and a simplification of the problem we describe.

We define a threat model as a set of pertubations, $\delta$ ($\delta:P\rightarrow P$) on the original packet sequences, $p_{o}$, to generate adversarial packet sequences $p_{a}$, ($p_{a}=\delta(p_{o})$). $\delta(p_{o})$ is adversarial if $f(\phi(\delta(p_{o})))\neq f(\phi(p_{o}))$, meaning that the perturbed flow will be misclassified by $f$ and $\delta(p_{o})$ preserves the semantics of $p_{o}$. Semantics are flow-level constraints on $x_{o}$ i.e., constraints that can be dependent on the whole packet sequence not just each packet indecently.

Our goal is to design a framework that generates packet sequences $p_{a}$ from $p_{o}$ such that $p_{a}$ are adversarial to $f$, semantic-preserving to $p_{o}$, and are within the capabilities of a threat model.

At first glance, adversarial ML (AML) seems to be the right approach for our problem definition and requirements. First, AML generates adversarial inputs against ML-based systems, typically in the image domain. Concretely, given a trained ML classifier with model parameters $\theta$, an original image sample $x$ with its ground truth label $y$, PGD perturbs $x$ to be an adversarial sample by iteratively running $x^{t+1}=\Pi_{x+S}\left(x^{t}+\alpha\operatorname{sgn}(\nabla_{x}L(\theta,x^{t},y))\right)$, $x^{0}=x$. $L(\cdot)$ is the target loss function, measuring the distance of the predicted label of $x_{t}$ and $y$, and $\Pi_{x+S}$ represents the capability of attack, which is usually defined as $\ell_{\infty}$-ball distance around $x$. The perturbation starts from the original sample $x$. In each iteration, the sample $x^{t}$ is moved by $\alpha$ following the direction of the gradient of $L$. After modification, the intermediate value is projected to be within the attack capability by clipping it to a predefined $\ell_{\infty}$-ball distance around $x$. In each iteration, we expect $L(\theta,x^{t+1},y)>L(\theta,x^{t},y)$ to have higher possibility to be misclassified. Second, AML is quite general, despite using gradient. In fact, there are AML methods that do not rely on a gradient or whose gradient information is inaccessible to the attacker, such as ZOO [7]. ZOO uses a combination of techniques, such as coordinate descent and importance sampling, to estimate the gradient for getting adversarial samples. Although it was initially designed for black-box attacks against NN-based models, it can be extended to attack other models such as Random Forest and XGBoost [5, 1, 2].

While AML can generate an adversarial sample and generalize at least across some models, AML does not provide realizability and semantics preservation. At a high level, the root cause of the problem is that AML finds adversarial inputs using backward reasoning (i.e., targets a change in the predicted label), but semantics preservation and realizability require forward reasoning, i.e., are defined on the input [9, 13]. In the image domain, this is not a problem as classifiers are end-to-end differentiable: images are directly fed into the ML model for classification, which is typically neural network-based. AML modifies input pixels directly and independently from each other (as they do not need to adhere to any rules beyond being in a range) to increase the loss informed by the gradient. Such techniques also include a budget to constrain just the pixel-to-pixel distance from the original with no other obligation.

However, in the networking domain, the traffic is often processed by the feature engineering module, which generates feature vectors that are fed to the ML model. While gradient information can still guide the modification of each feature to increase loss, it is unclear how to map modifications to the feature vector into the corresponding packet sequence as the feature engineering is non-differential and non-invertible. In fact, there is no guarantee that a perturbed feature vector would have a corresponding valid packet sequence as AML changes each feature independently, making contradicting decisions. For example, suppose an ML-powered intrusion detection system whose feature set includes both minimum and maximum packet inter-arrival times ($min\_iats$, $max\_iats$). Since AML does not understand the two features are dependent, in other words, that $min\_iat\leq max\_iats$ must always hold, it is possible for AML to decrease the value of $max\_iats$ and increase $min\_iats$ such that $min\_iats>max\_iats$. In this case, while the generated feature vector might be adversarial in that it might be misclassified by the intrusion detection system, it is not a realizable sample i.e., there is no packet sequence that would ever result in this feature set. In short, using the perturbed feature vectors is like defending against an impossible attack. While one can try to use a mask [30] to only allow AML to change a subset of the features that are not dependent, that would reduce the inputs that AML can find. Indeed, while AML can run with as few as a single feature there is no guarantee that the generated sample would be adversarial.

Even if the target model does not use (non-differentiable) feature engineering or if one somehow finds a packet sequence that is consistent with the AML adversarial feature vector, the packet sequence might still fall outside the capabilities of the threat model, violate any network constraints, or break the semantics of the original flow. For example, suppose the threat model only allows injecting packets, and flow duration $f_{1}$ and packets per second $f_{2}$ are two features extracted by the feature engineering module. The number of packets can be derived from $\#packets=f_{1}/f_{2}$. If $\#packets$ derived from the adversarial features is less than the number of packets in the original flow, even though one can find an adversarial packet sequence consistent with the adversarial features, it still falls outside the capabilities of the threat model. Similarly, for a deep NN that is directly accepting the packet sequence as input, the AML-generated adversarial sequence could change the interarrivals in a way that is impossible for any congestion control algorithm or in a way that makes the connection unusable because it is too slow in the beginning.

To address the inherent challenges in applying AML to MNCs, we integrate a formal-method component to ensure the realizability and semantics-preservation of the produced adversarial samples. Our first insight is to use logical formulas to model the feature-engineering process that turns packet sequences into feature vectors, and sequence-level constraints for MNCs with no feature-engineering, effectively encoding input dependencies. Further, we encode threat-model capabilities, networking rules and semantics into flow-wide or packet-level constraints and ask an SMT (Satisfiability Modulo Theories) solver to find a packet sequence that satisfies them. If the answer is “UNSAT”, the adversarial features are not realizable under the given threat model, semantic, and networking constraints. Observe that SMT supports both forward and backward reasoning, hence can connect the requirements for adversariness with realizability and semantics. Critically, this makes PANTS extendible to any definition of semantics.

As feature engineering modules, and sematics definition can be arbitrarily convoluted, a naive serial combination of AML with SMT will also fail because most of the AML-generated samples would not have a corresponding packet sequence that is realizable. Our second insight to address this challenge is that not all AML-generated features need to be honored for the sample to be adversarial. We design an optimization loop containing an AML component (e.g., PGD or ZOO), an SMT solver, and the target MNC, which iteratively checks whether a subset of the AML-produced adversarial features (or perturbations) can result in a realizable packet sequence until enough adversarial and realizable samples are collected. Instead of trying every possible subset of features, our third insight is to identify the most vulnerable adversarial features i.e., those that are both easy to tweak based on the threat model and important enough for the target ML model to affect its output. Critically, though, PANTS might opt out of constraints that make the generated samples adversarial but not constraints that make samples realizable or semantics preserving, which are always satisfied.

We provide an end-to-end view of how PANTS generates adversarial packet sequences, shown in figure 4. The procedure includes 8 steps, starting from feeding in the original packet sequence to getting the adversarial and realizable packet sequence returned by PANTS.

Step ①: Given an original packet sequence, we use the feature engineering module in the target application to extract its features. Step ②: After getting the features, PANTS uses the canonical AML methods (PGD, ZOO) to generate an adversarial feature vector (sample). The generation process interacts with the ML classifier to ensure the adversariness of the generated sample. Note that the generated adversarial features are not strictly required to be within a predefined $\ell_{\infty}$-ball distance around the original features since the capability of the attacker (threat models) for the networking applications is different from the image domain described in  §3.2. Step ③: Given the generated adversarial feature, PANTS runs an iterative process to find (multiple) adversarial and realizable packet sequences using a feature selector and an SMT solver. The feature selector first determines the importance of each feature in keeping the generated features adversarial. Step ④: The feature selector begins to construct a selected feature list by considering one new feature each time, starting by appending the most important feature and proceeding in descending importance order. Step ⑤: Given the selected features, we encode these features’ dependencies, threat model constraints, and networking constraints into formulas and query the SMT solver to find a packet sequence to satisfy them. Step ⑥: If the SMT solver returns “UNSAT”, i.e., it cannot find a packet sequence, the feature selector pops out the latest appended feature from the selected feature list. The iterative process continues to step ④ to append the next most important feature and repeat step ⑤ and ⑥ until the first $k$ most important features are considered. Step ⑦: The packet sequence(s) found by the SMT solver are guaranteed to be realizable since the threat models are encoded into the satisfiable formulas. Finally, we need to confirm the adversariness of these realizable packet sequences. We feed these packet sequences into the feature engineering module again to get the corresponding feature vectors. Step ⑧: We feed these feature vectors into the ML classifier to get the predicted label. We compare the prediction result with the ground truth label of the original packet sequence. If they are different, the realizable packet sequences found by the SMT solver are adversarial, and the feature vectors correspond to realizable packet sequences. Note that these packet sequences are very likely to be adversarial, given that the feature selector prioritizes the use of the most important features.

We leverage an SMT solver to help find adversarial packet sequences. We encode into formulas the feature dependencies, the threat model constraints, the networking and the semantics constraints. The SMT solver determines whether the resulting formulas are satisfiable.

We illustrate with an example. Assume an original network flow that only has three packets, p= $\{p_{1}^{+},p_{2}^{-},p_{3}^{+}\}$ where $p_{i}^{\{+,-\}}=(\Delta t_{i},l_{i})^{\{+,-\}}$. ${\{+,-\}}$ represents the direction of the packet, where $+$ means packets sending from the connection initiator to the receiver (forward packet) and $-$ vice versa. $t_{i}$ and $l_{i}$ represent the inter-arrival time and packet length for each packet $p_{i}$. Adversarial features dictate that the average inter-arrival times should be $k$, the maximum inter-arrival times should be $m$ and the average packet length is $j$. The threat model only allows to delay forward packets and append dummy payload to the forward packets. To preserve the semantics meaning, the accumulative delay cannot exceed 20% of the original duration. We thereby define four variables $\Delta t_{1}^{\prime}$, $\Delta t_{3}^{\prime}$, $l_{1}^{\prime}$ and $l_{3}^{\prime}$. They are representing the delay and the appended payload for $p_{1}^{+}$ and $p_{3}^{+}$. The perturbed flow p’ can be formulated as p’$=\{\bar{p}_{1}^{+},p_{2}^{-},\bar{p}_{3}^{+}\}=\{(\Delta t_{1}+\Delta t_{1}^{\prime},l_{1}+l_{1}^{\prime})^{+},(\Delta t_{2},l_{2})^{-},(\Delta t_{3}+\Delta t_{3}^{\prime},l_{3}+l_{3}^{\prime})^{+}\}$. Then, the formula for the SMT solver is

Equation (1), (2), (3) and (4) are forcing the adversarial packet sequences found to be consistent with the average (equation (1)), maximum (equation (2) and (3)) inter-arrival times and the average of packet length (equation (4)) in the adversarial features, maintaining the feature dependency. Equation (5) ensure that the packet sequence is complied with the threat model that it can only delay and append dummy payload to the forward packets. Equation (6) and  (7) formulate the network and semantics constraints.

The SMT solver’s efficiency in finding adversarial packet sequences is a critical factor for PANTS’ usability. Unfortunately, the SMT solver is often unable to find adversarial packet sequences within a reasonable time frame. The problem is that SMT solvers have scalability limitations and the given formulas can result in very high computational complexity. The complexity depends on many factors, such as the length of the packet sequence, the formulation of capabilities, the number of features to be solved, and the difficulty of solving each feature. We have already discussed restricting the number of used features and prioritizing the evaluation of the most critical features in §4.3. In this section, we focus on optimizing the packet sequence length and the formulation of capabilities.

We first set a time budget for the SMT solver. If the SMT solver cannot find packet sequences within a timeout window, we decrease the computation complexity by dividing a flow into several chunks evenly. The SMT solver is used to find the packet sub-sequences for each chunk. Since each chunk has fewer packets than the whole flow, it is more likely that the SMT solver will be able to find solutions within the timeout limits. After splitting a flow into multiple chunks, some features are difficult to be formulated strictly. For example, when working on a complete flow, it is easy to formulate the maximum packet length feature. After splitting the flow into multiple chunks, the maximum packet length can appear in any chunk. Strictly formulating this feature would require a complicated algorithm, which is in conflict with our motivation to decrease computation complexity and is unnecessary for our use case. To solve this problem, we consider three different types of features. For the first type of features, satisfying their formulas for each sub-sequence will likely result in satisfied formulas for the whole sequence. For example, such features are {max, mean, median, min, std} of {interarrival times, packet length} and number of {packets, bytes} per second. For these features, SMT solves their formulas per chunk. For the second feature type, the accumulation of each chunk’s feature values is equal to that of the whole flow. One typical example is the total duration. For these features, the SMT solver is finding sub-sequences in each chunk consistent with $\frac{feature}{\#chunks}$. For other features, such as the number of unique packet lengths, we are not aiming to be consistent with the whole-flow formulas when finding the sub-sequences for each chunk.

Finally, parallel computing can further speed up generation time, since the SMT-solving process for each flow/chunk is designed to be independent.

Figure 5 shows the sample generation speed after the discussed optimizations. PANTS achieves reasonable efficiency by taking less than 1s on generating a single sample in most of the cases. Generally, the generation speed doesn’t show notable difference between threat models, ML models and applications. However, for MLP and RF tested for the in-path threat model, the generation speed varies across different applications. The in-path threat model indicates a limited action space, making SMT harder to find solutions especially for those processing pipelines using feature engineering.

Adversarial training is often used to enhance the robustness of ML models. During authentic adversarial training, an AML method (e.g., PGD) replaces each benign sample with an adversarial one for the insufficiently trained version of the model. While useful, this method can hurt the accuracy of an ML model [17, 31] and only applies to NN-based models. To avoid these shortcomings, we adopt an alternative way of adversarial training, in which we augment the training dataset with adversarial samples generated for the sufficiently-trained model to fine-tune it. This methodology is model agnostic, meaning that it can work with any processing pipeline. We also find in §5.2 that this training methodology improves robustness without hurting the model’s accuracy.

VPN traffic detection is a challenging task for encrypted traffic classification. A well-functioning classifier is trained to determine whether the traffic is VPN. ISCXVPN2016[11] is one of the commonly-used labeled datasets, and includes both VPN and non-VPN traffic. The dataset contains more than $20$ million packets grouped into $8,577$ bi-directional flows. Of those, $71$% are non-VPN, and the rest are VPN flows. This is an unbalanced dataset, hence, we include both accuracy and F1 score for more comprehensive evaluation of the model performance.

Application identification (APP) is useful for characterizing mobile traffic from different applications and is crucial for network management tasks such as anomaly detection and resource allocation. UTMobileNetTraffic2021 [18] is a commonly-used dataset that includes mobile traffic from 18 popular applications (e.g., Dropbox, Google Drive, Facebook) under different user activities (e.g., Dropbox-download, Google Drive-upload, Facebook-scroll). It has multiple subsets representing different controlled user activities, such as scrolling the Facebook news feed. We choose one of these subsets that collects application traffic without placing any constraints on user activity. Similar to VPN detection, classification is applied to the bi-directional flows. The selected dataset contains 7,134 bi-directional flows.

Quality of Experience (QoE) inference arises as an important task for all stakeholders, such as ISPs and service providers, due to the rise in the use of Video Conferencing Applications (VCAs). By training various models, multiple QoE metrics can be inferred, e.g., resolution and video bitrate directly from packet traces. The state-of-the-art framework is trained and tested on a dataset called VCAML in a per-second granularity[33]. VCAML contains a large number of single-directional video conferencing flows from Google Meet, Microsoft Teams and Cisco Webex in different resolutions, bitrate, etc. We pick resolution prediction as the task to test our framework. The total number of samples is 37,274, which includes 11 different resolutions, such as 720p, 360p.

The features extracted by the feature engineering module for different applications are listed in the Appendix in  §A. All the datasets are applied with basic data cleaning. Any flow with only one packet is removed as calculating interarrival times requires at least two packets.

BAP is to train a generator to create perturbations for each packet. Since BAP [29] does not support applications that include non-differential and non-invertible components, we only consider it for TF and CNN but not MLP and RF. The original implementation of BAP can only support perturbing every packet, so we extend BAP by adding a mask during training and inference to ensure reliability (e.g., only act on forward packets). Although BAP supports per-packet constraints, it does not support flow-wide constraints such as those needed for semantics preservation.

We first compare PANTS with Amoeba and BAP in their ability to generate adversarial flows for various ML models, applications, and threat models. We run experiments five times and report the Attack Success Rate (ASR), i.e., the ratio of realizable, semantic-preserving and adversarial samples / total original samples for each technique²²2While PANTS generates multiple adversarial samples per original, for the ASR calculation each original sample can contribute at most one adversarial.. Figure 6 shows the results of ASR. PANTS achieves 35.31% ASR in median while only 19.57% for Amoeba and 10.31% for BAP. Among these three methods, BAP performs the worst in part due to its inability to respect semantics i.e., flow-level constraints.

Compared to Amoeba, PANTS is 51% more likely to find adversarial samples for the end-host threat model in median and 2X more likely for in-path attacks. We observe that Amoeba struggles to find adversarial samples when for an attacker that is in-path. Concretely, for the QoE inference, the ASR for Amoeba is around 0.3% for in-path threat model while PANTS is 37%. PANTS is particularly more effective with the weak threat model compared to Amoeba, showing its more comprehensive nature.

Importantly, PANTS’s ASR is more consistent across runs in all the test cases. This is important as an operator is unlikely to trust the assessment of a tool that reports a different ASR in every run i.e., varying levels of vulnerability for a given MNC. Concretely, the average standard deviation across all the test cases is 0.77 for PANTS, while for Amoeba is 3.12 and BAP is 2.61. For some specific test cases, Amoeba and BAP show ASR with relatively large fluctuations. For example, Amoeba’s ASR on attacking TF ranges from 76.15% to 93.45% and BAP is from 43.89% to 54.25% while PANTS’ ASR only ranges from 99.75% to 100.0% as we observe in Fig. 6(a). The large variance in Amoeba’s results stems from a well-known problem in RL: its inherent instability of training. For PANTS, randomness only comes from the adversarial features generated by PGD and its timeout mechanism. PGD might generate different adversarial features in different runs due to random restarts, which are used in MLP and RF [27]. The time budget in SMT also causes minor randomness by the CPU’s processing speed fluctuation. It turns out that randomness caused by these two factors are small.

As Amoeba is more general and results in higher ASR compared to BAP, we continue our evaluation on robustification with Amoeba only. Concretely, we robustify the target ML models by iteratively fine-tuning them on a combination of the original dataset with its adversarial counterpart generated by PANTS and Amoeba as the way described in §4.5.

We found that in most of the cases, 3 iterations of robustification are enough when using PANTS to robustify an MNC. The robustified MNC is able to defend the adversarial inputs that are generated by PANTS itself, Amoeba and BAP. For TF in the application APP and QOE under end-host threat model, we train for 5 iterations to reach a desirable ASR. Figure 7 shows a trajectory of ASR when a vanilla MNC is in iterative robustification using PANTS’ samples. During the iterative training the ASR (calculated using PANTS to generate adversarial inputs) drops, meaning that the model is getting more robust. Iteratively training MNCs using adversarial samples generated by Amoeba follows a similar but less dramatic trend. The results are shown in Appendix in the fig. 12.

After robustification, PANTS generates multiple adversarial samples for each benign counterpart in total to cover diverse adversarial samples. Using PANTS, we generate around 1.16-180 samples for each benign sample during robustification based on different threat models, applications and ML models. We also include non-adversarial samples generated by PANTS (not just adversarial). This is beneficial for training because the non-adversarial samples still have a higher loss than their original benign counterpart, being closer to the decision boundary, even though they can be correctly classified. Including those samples can help form a more refined decision boundary. In any case, including non-adversarial samples in the training set offers Amoeba more data to train.

We compare four ways of robustification, ①: authentic adversarial-training, and iterative augmentation using ②: PGD’s samples, ③: PANTS’ samples, ④: Amoeba’s samples. ②, ③ and ④ share the same iterative way of robustification and the only difference is the method to generate adversarial samples. PANTS and Amoeba generate adversarial, semantic-preserving, and adversarial samples while PGD just generates adversarial samples. Figure 8 shows the accuracy and ASR of the robustfied ML models against end-host attackers under the application APP. Adversarially-trained models are robust against attacks from PANTS and Amoeba, but suffer from low accuracy. In contrast, iterative augmentation doesn’t drop the model accuracy, but not all of them can improve robustness for the model. PANTS-robustified models are more robust against both PANTS and Amoeba compared to the vanilla one. But Amoeba-robustified models can only be robust against Amoeba and PGD-robustified cannot be robust against any of the attacks.

After robustification, we use again PANTS, Amoeba and BAP to re-attack (generate adversarial inputs for) the robustified models $M^{\prime}$ to re-evaluate models’ vulnerability. We train a new agent for Amoeba on $D_{train}^{\prime}$ to fool each robustified model $M^{\prime}$. Figure 9 shows the ASR using PANTS, Amoeba, or BAP to find adversarial samples from the vanilla, PANTS, and Amoeba robustified models for the end-host threat model. ASR for the in-path threat model is shown in figure 11 in the Appendix. These results show that PANTS-robustified models are generally robust against the attack from PANTS, Amoeba and BAP, while Amoeba-robustified models are only robust against Amoeba. Table 2 shows the robustness improvement when robustified by PANTS and Amoeba and evaluated with PANTS, Amoeba and BAP. Robustness improvement is defined as $(ASR_{vanilla}-ASR_{robustified})/ASR_{vanilla}\times 100\%$. PANTS-robustified models demonstrate 52.72% robustness improvement on average while Amoeba-robustified ones only demonstrate 21.76%. PANTS-robustified models can have 142% more robustness improvement than Amoeba-robustified ones.

Prior works, especially in RL, have found that adversarial training can only defend a model against adversarial samples generated by the same generation process as the one used in training [37]. However, we found that augmenting with a large number of adversarial, realizable and semantics-preserving samples has the potential to robustify the model against multiple different attacks. That is the case because the space of adversarial, realizable, and semantics-preserving samples (generated by PANTS) is smaller than the space of samples that are only adversarial.

PANTS-robustified models demonstrate considerable robustness improvement against PANTS, Amoeba and BAP. However, we do not claim that PANTS-robustified models are robust against any possible attacks. First, the ASR for PANTS-robustified models doesn’t drop to 0, meaning that these attack methods can still find adversarial samples against the robustified models. Second, as an empirical way of robustification, we don’t have a theoretical guarantee that the robustified models are robust against the full spectrum of perturbation. Figure 11(f) shows that there still exists at least one such that the PANTS-robustified model doesn’t provide enough robustness. Still, this work presents a clear, extendible move towards the right direction.

There are use cases in which a network operator will not have access to the MNC,   e.g., because the MNC is designed by a third party and its details are secret. While this is not our target setting, we show that transferability of adversarial samples PANTS might be able to help the network operator e.g., by providing examples in which the target MNC misclassified or by providing potentially intresting test cases for the MNC designer.

Transferability is defined as the ratio of adversarial samples that are generated for a given implementation of ML classifier that can also be adversarial against another one. Concretely, suppose we evaluate transferability of the adversarial samples generated from $M_{A}$ to $M_{B}$, we first construct the test set $X_{test\_A\_B}^{\prime}\subseteq X_{test}$ that both $M_{A}$ and $M_{B}$ can correctly classify. Then we use PANTS to generate adversarial samples for $X_{test\_A\_B}^{\prime}$ using $M_{A}$ and evaluate its adversariness against $M_{B}$. Figure 10 shows the transferability result between MLP, RF, TF and CNN for end-host threat model tested for the application APP. Most of the transfer ASR is above 50%, showing that PANTS can be useful even without full access to the MNC implementation. To benefit from it the operator would need to train a proxy model and find adversarial samples against it using PANTS. According to Figure 10, the adversarial samples against the proxy model are likely to be adversarial against the target one. Note that the transfer ASR between different models cannot be compared directly. For example, the ASR of transferring adversarial samples from CNN to RF is 70.44% while directly attacking RF (transfer ASR from RF to RF) is 66.5%. This doesn’t necessarily mean that transferring CNN’s adversarial samples to RF is more effective than directly attacking RF directly attacking RF because transferring from CNN to RF is tested on $X_{test\_CNN\_RF}^{\prime}$ while directly attack RF is tested on $X_{test\_RF\_RF}^{\prime}$. As $X_{test\_CNN\_RF}^{\prime}\subset X_{test\_RF\_RF}^{\prime}$, the two tested datasets may not be identical.