Not All Datasets Are Born Equal:
On Heterogeneous Data and Adversarial Examples

The susceptibility of learning algorithms to adversarial input manipulations has puzzled researchers in recent years, leading to extensive research work. Early work on adversarial attacks was demonstrated using learning tasks that were popular at the time, such as spam filtering [1, 2] and malware detection [3]. Notably, homogeneous, bag-of-words feature representation was used in all relevant use cases. Later on, as deep learning has become prevalent, the vast majority of papers published in recent years have discussed attacks and defenses in the context of neural networks. Naturally, these works have focused on input domains where deep learning excels such as image and audio, where data representation remains mainly homogeneous and input spaces are continuous. A significant amount of research effort was put into computer vision applications and related tasks, e.g., image segmentation [4], face detection [5], and more specifically, image classification [6, 7, 8, 9, 10, 11]. Other domains include audio [12], and the exclusively discrete domains of malware represented by binary vectors [13, 14] and the classification of sparsely encoded text [15, 16].

In contrast, there has been little to no research on heterogeneous input spaces in the context of adversarial examples. The reasons for this range from the intrinsic diversity of feature data types, to the relatively underexplored robustness of the tree-based models that are most associated with such data [17, 18, 19]. The few attacks that were demonstrated in the context of heterogeneous input domains have focus on exploiting the architecture of the learning model, while ignoring the effects of the manipulation on the validity of the data itself. Another possible explanation is that machine learning research in recent years was almost entirely focused on deep learning based methods, leading researchers to study data taken from high-dimensional, homogeneous and continuous distributions, where deep learning typically demonstrates improved performances over traditional approaches [20]. Conversely, tabular data is drawn from low-dimensional, heterogeneous, and largely discrete input domains. More specifically, tabular data has the following characteristics: (a) the input often includes a combination of nominal, ordinal, and real-valued features, (b) different value ranges are associated with different features, (c) missing values are common, (d) some features are considered immutable in the context of an attack, and (e) complex cross-feature interactions define valid and anomalous feature combinations. Examples for domains where the data is normally heterogeneous include healthcare, real estate, and financial applications.

In this paper, we study the process of crafting adversarial examples for heterogeneous tabular data. There are several challenges that are unique to this case, both from the perspective of the adversary and challenges that are associated with the optimization procedure of finding adversarial perturbations that must comply with the heterogeneity. In contrast to audio or language processing applications that are considered digital, machine learning tasks on heterogeneous data are usually related to real-life scenarios, and the relevant features describe tangible information, such as a person’s salary. Therefore, an adversary that targets heterogeneous data would like to minimize the number of modified features (i.e., a minimal $\ell_{0}$ perturbation), as opposed to minimizing other frequently used distance metrics (e.g., $\ell_{1},\ell_{2},\ell_{\infty}$) which are compromised by the vastly different feature scales. Additionally, some input features must be considered immutable in the context of the attack, as it is impossible (or at least very difficult) to change their values in real life. As an example consider features that denote a person’s height, credit history, or residential address. This implies that an adversary must also address the real life feasibility of the identified adversarial perturbations. Data heterogeneity lead to optimization challenges as well. A generic attack algorithm must preserve the consistency of the adversarial examples in terms of feature correlation, and to maintain the various types of features (e.g., integers, positives, real numbers).

Our approach for perturbing samples from heterogeneous input spaces addresses each of the aforementioned challenges, while being agnostic to the target learning algorithm and the data domain. First, we formulate the mathematical modeling of the validity of the data, in terms of its consistency (i.e., preserving feature correlation) and feasibility (i.e., keeping immutable features intact). Then, we aim to capture the complexity of the heterogeneous input space by using a data transformation function, whose purpose is to embed the heterogeneous data into a continuous latent space and lead to more consistent adversarial examples. This function encompasses the correlations between the features and thus allows us to use gradient-base optimization techniques to identify adversarial examples without harming the consistency of the input. Using these tools and techniques, we define a general optimization problem for finding valid adversarial examples of heterogeneous tabular data, with the ultimate goal of applying them in real life. Finally, we propose a clear implementation of our approach and its various components, based on neural networks and methods from metric learning. We test our approach and implementation using three datasets from different content domains and a variety of target learning algorithms.

Our results suggest that machine learning models in domains with heterogeneous input spaces are just as prone to adversarial manipulations as those used in continuous homogeneous domains. This implies that further research is required to understand the role played by data, as opposed to the model or algorithm, in the context of adversarial susceptibility.

Tree-based models achieve state-of-the-art performance in learning tasks that involve heterogeneous tabular data [17, 19, 18]. The robustness of such models has been extensively researched, including attacks on decision stumps or trees [21, 22, 23], random forests [24, 23], and gradient boosting machines [25, 23]. Similarly to these works, prior studies on traditional non-tree-based models were either too specific to the domains they were demonstrated in [1, 26, 27] or tailored to a certain class of learning algorithms (e.g., SVM or linear models in general) [2, 28]. The most prominent attack methods today target neural networks [6, 7, 8, 9, 10]. The vast majority of these attacks relate to a more general technique based on the network’s gradient or its approximation. However, in the case of nominal features which are common in tabular data, the derivative with respect to such features (i.e., $\frac{\partial\mathcal{L}}{\partial x_{i}}$ for a loss function $\mathcal{L}$ and a nominal feature $x_{i}$) cannot be correctly interpreted, as it is arbitrary at best. As a result of this and other issues, all previously mentioned research either lacks generality or could not be adapted out-of-the-box to applications with heterogeneous input space. We aim to bridge this gap by extending the renowned adversarial deep learning tools to create a neural network-based implementation of our generic approach.

Recently, there have been some attempts at manipulating neural networks trained on heterogeneous tabular data. Sarkar et al. [29] performed a number of attacks on a credit score classifier and demonstrated vulnerabilities in financial applications. However, the experiment only included continuous features despite the heterogeneous data domain, and the evaluation metrics are not well-suited to a real-life adversary. Ballet et al. [30] presented a real-life data-oriented approach that could be adapted to domains other than finance. It employs the means to (1) preserve the data types, by clipping, and (2) reduce the perturbation’s perceptibility, by incorporating feature importance in the attack objective in an attempt to mimic the decision process of a domain expert. Their method and experiment did not appropriately reflect the heterogeneity of the data, since categorical features were treated as continuous, and nominal features were discarded altogether. Moreover, the evaluation of the perturbations was based on the $\ell_{2}$ norm, which is misrepresentative given the significant variability in scales of the features, and the feature importance (i.e., knowledge of a domain expert) was improperly modeled by using the Pearson correlation with the target variable.

A primary goal of our work is to cover all aspects related to the feature space challenges of heterogeneous tabular data. The following works integrate components related to covariances or outliers with the intent of preserving feature correlations or feature types. An early work [3] in the adversarial learning domain takes the underlying distribution of the data into account in their proposed attack. Their method includes modeling the a priori estimate (i.e., $p(x\,|\,y)$ for a data point $x$ and its label $y$) by using a kernel density estimator as an additional component in their attack objective. However, this method was not evaluated on heterogeneous input spaces and therefore lacks additional related aspects that are essential. Ustun et al. [31] formulated an integer linear programming (ILP) problem to find perturbations that do not include immutable features and are consistent (i.e., preserving the data’s structure under its various mathematical constraints) by adding constraints to the problem definition. Kanamori et al. [32] extended this work by incorporating non-linear statistical elements that correspond to the empirical distribution of the data, in a mixed ILP-based optimization instance. Their method aims to create realistic perturbations that conform with feature correlations and outlier values, by adding approximations of the Mahalanobis’ distance [33] and the local outlier factor [34] to their objective function and optimization constraints. The context of the two last-mentioned works is counterfactual explanation methods [35] of machine learning models used to provide recourse to users who wish to gain access to the factors underlying the model’s decision (e.g., a credit scoring application where customers try to improve their score). Contrarily, our work focuses on an adversarial setting and related topics.

The approach we propose in this paper complements previous work by weaving the various concepts we discussed so far into one exhaustive formulation and implementation, and therefore the approach addresses all possible challenges that arise from crafting adversarial examples for heterogeneous data. To the best of our knowledge, we are the first to introduce a comprehensive and general adversarial attack method that is agnostic to the target learning algorithm and focuses on truly heterogeneous tabular input space.

In this section, we describe our approach using the following notations. The attacker targets a machine learning model $M:\mathcal{X}\rightarrow\mathcal{Y}$. For brevity, we assume $\mathcal{X}$ is a discrete input space with $d$ discrete random variables. The label space is denoted by $\mathcal{Y}\subseteq\{0,1\}^{m}$ where labels are one-hot encoded.

Let $X_{1},\dots,X_{d}$ be the random variables that represent the features in the data, which have the discrete distributions $\mathcal{F}_{1},\dots,\mathcal{F}_{d}$, such that for each, $1\leq i\leq d$, $X_{i}\sim\mathcal{F}_{i}$. Therefore, an input vector $X=(X_{1},\dots,X_{d})$ follows the multivariate joint distribution $\mathcal{F}$, as in $X\sim\mathcal{F}$. We note that $X_{1},\dots,X_{d}$ are not necessarily independent or identically distributed. It would be even safer to assume some level of dependency between the features as naturally prevails in tabular data.

Given a benign sample $x\in\mathcal{X}$, with the corresponding ground true label $y\in\mathcal{Y}$, a maximal perturbation size $\lambda$, and a context dependent distance metric $\mathcal{D}$, the adversarial goal is to find an adversarial example $x+\delta=x^{*}\in\mathcal{X}$ such that $\mathcal{D}\left(x^{*},x\right)<\lambda$ and $M(x^{\ast})\neq y$.

We assume that the attacker has access to the training data $\mathbb{X}\subseteq\mathcal{X}$ used for constructing $M$. Although our approach introduces an untargeted attack, it can be easily modified to fit a targeted attack.

The surrogate and target models are evaluated using the standard metrics relevant to the task: the receiver operating characteristic-area under the curve (ROC - AUC) for imbalanced classification and the MSE for regression. The performance of the models presented in Table 2 shows that the GBM model consistently outperforms every other model in every use case and that its performance is also very close to highly-ranked submissions in the corresponding Kaggle competitions (apart from the Airbnb which is not part of a Kaggle competition). It is important to note that our surrogate model does not fall too far behind the GBM model, which means that the embedding sub-network did not affect the overall performance too negatively.

As we have established, the metric most relevant to the case of heterogeneous data and real-world use cases is the $\ell_{0}$ norm, as the other norms are compromised by the significantly different scales of features. However, we also decided to include the $\ell_{1}$ norm in our analysis for just numeric features in the perturbations, in order to obtain some understanding of how these features are perturbed in cases in which there are unusually big changes. In addition to the $\ell_{0}$ norm, we also looked at its percentage of the entire dataset’s features, as the number of features varies between the use cases. An example of the need to analyze the percentage can be found in the Airbnb use case where the value of the average total $\ell_{0}$ distance is $5.28$, but when incorporating the total number of features in the dataset, which is $157$, we get an average relative $\ell_{0}$ (i.e., total $\ell_{0}(\%)$) of only $3.36\%$. In Table 3 we can see that in every use case the bulk of the average perturbation, are changes in categorical features. Moreover, since the Airbnb dataset contains considerably more categorical, which are mostly binary features (representing the existence of amenities in the property), the difference is even more noticeable, with $4.33$ of the total $\ell_{0}$ distance of $5.28$ being changes in categorical features. Fig. 2 presents histograms of the $\ell_{0}$ norms of the perturbations, which are highly right-skewed and indicate that the vast majority of the adversarial examples include changes to only 1-2 features. The success rate is consistently high throughout the use cases, with nearly all of the samples in the attack set becoming adversarial examples.

In addition to attacking the surrogate model, we examine the transferability of the adversarial examples to GBM, random forest, and decision tree models. As described in Algorithm 1, the iterative feature selection is based on the gradients of the surrogate network. When targeting models other than the surrogate, this selection process can be adjusted so that the impact of each feature is represented explicitly by the model, rather than implicitly by a surrogate’s gradients. We changed it to be the feature importance of the tree-based models we target, namely the information gain of each feature. In Table 4 we report the transferability rates of the adversarial examples for each of the target models, as well as the rates of the adjusted attack (i.e., when feature selection is based on the target model’s feature importance) in parentheses. The base transferability rates in all use cases and models is already high, however the adjustment made evident and substantial improvements consistently throughout our experiments.

In our experiments, the specific compositions of the various datasets (i.e., the size of groups of feature types) and the distribution of the constraints (as described in Table 1) have a direct effect on the adversarial optimization process. Generally, we observed that heterogeneous input spaces involve several constraints and pose many challenges to the adversarial susceptibility of learning algorithms. Our results show that not only is it possible to overcome such challenges and attack neural networks trained on heterogeneous tabular data very successfully and effectively, but that the transferability property of the adversarial examples generated is rather persistent in this unique case as well.

In this paper, we explore the role played by data in the susceptibility of machine learning models to adversarial manipulations. We introduce a general approach that formalizes the generation of valid adversarial examples for heterogeneous tabular data, and provide an implementation of such an attack in an optimization-based context. Our approach is focused on an $\ell_{0}$-oriented adversary and her ability to apply the adversarial perturbations in real life. To the best of our knowledge, this is the first study to present an automated attack for heterogeneous tabular data. Our results support the possibility of successfully attacking previously unexplored domains with heterogeneous input space.

Future work may place greater emphasis on the important aspect of preserving feature correlation and improving our implementation, as well as developing ways to properly evaluate the success of such preservation. Additional implementations for our approach should also be explored, mainly different embedding functions and other methods for crafting adversarial examples. Advancement of mitigation methods could have more potential in the case of heterogeneous data than in the general case, as the baseline naive approach would be to limit the training to immutable features.

We believe that our work has identified an important research direction in the field of adversarial learning and broadens its scope beyond the main applications of computer vision.