Hybrid Deep Learning Model using SPCAGAN Augmentation for Insider Threat Analysis

Insiders are individuals such as employees, contractors, or business partners of an organization who are trustworthy sources, have legitimate access rights, and execute suspicious behaviors within the organization, either purposefully or accidentally. Unintentional activities sometimes occur due to negligence or a lack of understanding of security policies. Although insider threat analysis has been studied for many years, the emergence of new approaches is limited due to a lack of real-world data and difficulty finding an effective solution.

Nowadays insider threat analysis attracts wider attention due to the increase in frequency of attacks and the advent of promising data analysis techniques. Existing works helped to understand the exhaustive range of methods applied from various perspectives. To list a few, it includes statistical metric based solutions [9], machine learning [10], image-based analysis [11], blockchain-based analysis [12], natural language processing (NLP) approaches[13] and dealing with the insider attack in several critical application areas like health care [14] and Internet of Things [15]. There is an upward trend towards using machine learning and deep learning based solutions for insider threat analysis.

In the survey [16], Homoliak et al. presented a classification of insider threat research publications and promising study directions. The authors identify the use of GAN for adversarial classification for enhanced threat detection and data augmentation while recommending possible research directions.

Existing works use various data generation and augmentation strategies in insider threat detection. The method followed in [17] based on the email communications create a new set of data, Enron⁺, followed by anomaly detection using Isolation Forest algorithm. The scenario-based classification in the work [18] considered the problem as three different binary classification problems and used Random Over Sampling (ROS) for data augmentation under various sampling ratios. The method gave good results for Scenario 2 on Random Forest whereas it could not perform satisfactorily on Scenarios 1 and 3. Spread subsample, a method that selects the random samples to fit in the memory by balancing the skewed class distributions is mentioned in the paper [19]. An ensemble strategy combined with data adjusted XGBoost model for insider threat detection proposed in [20] gave promising results. All these works performed supervised binary classification for insider detection. In the paper [21], the authors considered class labels also in the data generation process which helped in generating more meaningful data.

Unlike other synthetic data generation methods, GANs can create realistic data samples that significantly reduce data imbalance and ultimately overfitting. Since the network uses the original data and noise to create synthetic data, it can be considered a transformation approach where generated data points mimic the real data distribution. GANs are very popular with image processing applications. Very recently, GANs have been applied in cybersecurity [22], [23] which is a source of heterogeneous data in structured and unstructured form.

In the paper [22], the authors used generative adversarial networks in intrusion detection. Lee and Park [23] proposed a method for detecting intrusion using GAN and random forest algorithm. The model was built to solve overfitting. The experiments show that the model achieved its goal and performed well in imbalanced datasets. The authors have proposed a conditional GAN [24] for insider threat detection, which proved to be effective for multiple attack detection with many minority classes.

Most of the existing GAN research concentrates on unstructured, continuous data such as images. However, real-world datasets used for classification purposes are mostly tabular and include numerical and categorical attributes. GAN-based augmentation methods are gaining popularity [25], [26], [27]. GAN-based approaches for modeling tabular data have recently been developed, whereas GAN-based approaches for numeric data are still in the budding stage.

Tabular data synthesis comprises a variety of methodologies based on the types of data. It models a joint probability distribution of columns in a table to create a realistic synthetic table. However, due to the limitations of several of these models, such as distributions and processing issues, high-fidelity data synthesis has proven to be challenging. Several GAN-based data generation algorithms for generating synthetic tabular data, notably for healthcare records, have been developed in recent years. These methods can be applied on the numeric feature vectors to generate more samples in various application domains. In the paper [28], the authors mention the various research directions possible in GAN-based analysis; the use of regularization and the need for modified architectures are a few among them.

Recently, deep learning methods gained attention in insider threat analysis. Yuan and Wu [29] performed an exhaustive review on the challenges and the potential research directions for deep learning in insider threat analysis and pointed out extreme class imbalance as one of the key challenges. Deep neural networks (DNNs) are a popular forecasting tool due to their versatility and ability to represent non-linearities despite the fact they tend to overfit and hence are not recommended for small data sets [30]. They also don’t measure the degree of uncertainty in the projections.

Bayesian Neural Networks (BNNs) can overcome the above mentioned problems using prior distributions on the parameters of a DNN model and expressing uncertainty about the predictions in the form of a distribution [6]. Rather than representing each estimated parameter as a single point, they express it as a distribution. The field of Bayesian Deep Learning, which uses the Bayes technique for neural network models, is fast-growing [31], [32].

Ensemble techniques [33], [34] combine multiple machine learning algorithms to provide better predictive performance than a single machine learning classifier. The primary idea behind an ensemble approach is to mix various machine learning algorithms to make use of the strengths of each one to create a more robust classifier. Ensemble techniques are beneficial when the problem can be divided into sub-problems, and each sub-problem is given to one of the ensemble’s modules. Each module can comprise one or more machine learning algorithms, depending on the ensemble approach’s structure. The most challenging part of adopting ensemble techniques is deciding which collection of classifiers to use to make up the ensemble model and which decision function to use to integrate the outputs of those algorithms.

Inspired by the success of GAN as an oversampling method for data augmentation, this article proposes a GAN-based data augmentation insider detection model referred to as SPCAGAN. The model aims to reduce class imbalance issues from the data level. First, SPCAGAN is used to generate specified minority classes followed by merging the generated data into the original training data to create a new training dataset. The proposed model increases the sample diversity and alleviates the class distribution imbalance of the dataset. Second, the new training dataset is used to perform insider threat detection using hybrid Bayesian neural network classification algorithms. We explain the proposed approach in detail in the forthcoming section.

Insiders are entities that perform harmful activities from within the organization using various resources to which they have access. Insider attacks can be intentional or unintentional, but the consequences can be severe. Insider activities typically leave a minimal trace in the audit data trails. A variety of behavioral characteristics can identify malicious insiders. Copying significant volumes of data to external/USB drives, transferring confidential material to own personal account via emails, abnormal or untimely access patterns on official resources, and using unlawful, abusive, or unethical language in emails or papers are only a few of them.

This section deals with the feature space generation for the insider threat analysis. Each user is identified as an insider based on the entire activity log instead of those related to the particular scenario. User behavior profiling is one of the significant steps in insider threat detection, and this work uses context-based user profiling, where all the features contribute to user behavior. Scenario refers to the sequence of activities that result in a malicious event. The distinct feature sets identified for each scenario are merged to form a single feature set representing context-based features. This makes the feature space easy to scale in cases of new scenario identification.

Growth in technology led to replacing traditional workplace systems with emails, electronic files, websites, and information sharing mediums like external devices, demands for changing conventional security methods to track the employees. Recently, artificial intelligence-based solutions are more popular due to their capability to study historical and current data patterns and develop valuable and usable learning to deter and prevent these kinds of activities.

The various user activities used to represent user behavior are derived from log files. The user activity logs obtained from the logon-logoff details, file access history, external device usage patterns, email communications, and the web browsing history files are pre-processed. We try to incorporate features from each resource mentioned above. Features can be generated based on the scenarios defined in the CERT standards [35].

The features extracted from the resource files are used for threat intelligence techniques. The existing works use numerous feature sets engineered from various perspectives, which are being used as in the machine and deep learning models. As the number and values being used in the features significantly impact the learning models, we performed a step-by-step analysis of the number of features for the user behavior profile. Using the features from the paper [18], we added features from different resources followed by feature selection to derive the effective feature list.

The features newly introduced in this work include the sentiment analysis of the text available in the resources like email and web access files helps in understanding the user sentiment in the communication. Moreover, we included the psychometric details of the users, which hints about the user resource usage in various contexts. We extended the feature list by adding sentiment analysis, holiday details, in-depth feature identification from the email, and web browsing history. Later, we eliminated the less meaningful and dependent features using the feature selection methods.

In the review paper, Guyon and Elisseeff [36] presents the need for feature selection that acts as a catalyst in the model building in many ways like reducing the training time, helping in selecting a simpler model, reducing the dimension, reducing the chance of overfitting thereby enhancing the model generalization. Non-multicollinearity of predictor variables is assumed in the most linear analysis, which means that pairs of predictor features cannot be correlated. The removal of correlated features is a fundamental feature selection strategy. Multi-correlated features also indicate duplicate features in the data; therefore, removing them is an appropriate starting point when minimizing potential dimensions.

We can address the challenges of feature redundancy and predictor collinearity by eliminating correlated features, thereby only preserving one of the groups of observed associated characteristics. Finally, the feature list consists of 47 features from all the available files. The feature engineering detailed in this section is not restricted to any particular dataset but can be used for any data with minimal adaptation depending on the availability of the log files. Table I gives the details of features extracted from the resource access logs.

The feature set comprehensively describes each user’s daily resource usage pattern, which results in a highly imbalanced class distribution due to the rarity of insider attacks. The data samples are labeled based on the scenario from the user’s activity pattern. In this work, our main aim is to use a generative model to balance the data distribution and build a hybrid learning model for anomaly detection. The following section explains how generative adversarial networks are utilized to generate meaningful data samples that help to reduce the skewed class distribution.

Data augmentation means expanding the quantity and variance of a dataset used to train a machine learning model to improve generalizability and gain a better understanding of the underlying distribution of the training data. The space that represents the training data distribution can be viewed as the manifold of a class learned by a classifier. Here, we introduce the generation of adversarial data samples for insider threat analysis.

When presented with massive, multidimensional data sets, it is frequently beneficial to identify or impose some structure on the data. The smallest number of parameters M required to explain the data is one such structure. This value M is the data set’s intrinsic dimensionality (ID), or more precisely, the data generation process. According to the geometric interpretation, the complete data set is contained within a topological curve of M or fewer dimensions. It is required first to compute the intrinsic dimension of the data set to build a mapping connection from the original data to the reduced feature space. This mapping can be linear/nonlinear and explicit/implicit based on the domain where it is used.

Approaches for estimating IDs that have been created in the past can be categorized into two broad categories: local methods and global methods. Local methods calculate the topological dimension of data using data from sample neighborhoods. Global methods, such as projection, multidimensional scaling, fractal-based analysis, and multiscale analysis, use the complete data set, considering that it is contained within a single manifold of defined dimensions.

The projection method involves projecting data into a low-dimensional space and then confirming the low-dimensional representation of data to obtain the ID. PCA is a standard projection approach that counts the number of significant eigenvalues to find the ID. The intrinsic dimensionality of a data set may be critical in determining the characteristics of classifiers applied to it and, as a result, in selecting the best classifier. These techniques are classified as linear or nonlinear. Principal component analysis (PCA) is a commonly used linear technique discussed below.

S_PCA, devised by Krzanowski [37], is a method for assessing the similarity of two datasets using a PCA similarity factor. The PCA Similarity Factor (S_PCA) compares two matrices with the same number of columns but not the same number of rows. S_PCA first computes the principal components of each matrix and then uses heuristics to choose the first $k$ principal components. For example, the top $k$ principal components are chosen because their variances account for 95% of the overall variation. The SPCA algorithm then calculates the similarity of the first k principal components. The PCA Similarity Factor, S_PCA, between two matrices, A and B, is determined using the angles between principal components and is defined as follows.

where L and M are the loadings matrices of the first k principal components of A and B, respectively. The coefficients of the linear combination of the original variables from which the principal components (PCs) are derived are known as PCA loadings. This value can be used to compare the overall similarity of two spaces and is clearly between $k$ and $0$, where k is the number of PCs. The range of S_PCA is between $0$ and $k$. The advantage of using this method is that the data size is not so important; it is based on the principal components and the distance between those vectors in the subspace. Hence, can be more reliable than correlation-based ones. We exploit this metric for regularizing the GAN training.

Adversarial training [4] is a helpful approach for making neural network models more robust. This technique’s core concept is to apply well-known attack methods to generate adversarial samples during the model training stage. Adding the adversarial samples to a training set and retraining until a new model that is resistant to perturbations is built. The robustness and accuracy of the newly created model can be improved using this method. However, the synthesis of different classes of adversarial samples, which increases the amount of data in the training set, can improve the model’s generalization and accuracy.

A generative adversarial network (GAN) is composed of two neural network modules, a generator and a discriminator. The objective function of the real GAN commonly referred as vanillaGAN is:

In the above equation, z represents random noise, p$z$ represents the distribution of noise samples z, p$r$ represents the distribution of real data x, p$g$ represents the distribution of attack samples generated by G, G(z) represents the pseudo data generated by generator G, and E(.) represents the expected value. The two networks compete against each other and are iteratively optimized, with D maximizing the accuracy of differentiating data sources and G generating more realistic fake samples to fool the discriminator D.

Many GAN variants like CGAN [38], WGAN-GP [39], ACGAN [8] are used in many applications. The vanillaGAN framework, on the other hand, lacks control over the generated data because it is an unconditioned generative model. In other words, the labels for the created images cannot be predicted. Mehdi et al. [38] presented the conditional generative adversarial network (CGAN) as a way to incorporate label information into the training process and train a generator that produces data from the given class. To produce data under the desired conditions, CGAN was proposed.

The CGAN generator generates data with the specified condition by taking a pair of latent vectors and condition vectors as input. The generator estimates the distribution of P$(X|y)$ and the discriminator learns to estimate D(X, y) = P(fake|X, y). Following that, ACGAN was proposed as an enhanced version of CGAN. ACGANs are also capable of controlling the output with additional input, but its architecture is more complicated.

The stability in the training process and its ability to generate high quality synthetic data motivated us to chose ACGAN for inside threat data augmentation. However, for insider threat assessments, the direct application of ACGAN for data augmentation does not give adequate results as seen in the results explained in Section IV-C. Different regularization procedures can be considered. We describe a variation that strongly integrates a regularization factor into the optimization technique as part of our attempt to synthesize the features of real-world data.

As with any GAN network, the ACGAN has a generator and a discriminator. Like CGAN, the ACGAN’s generator model takes a point in the latent space and a class label as input to condition the synthetic data generation process. Unlike the CGAN discriminator, which accepts both data and class labels as input, the ACGAN discriminator accepts real and synthetic data samples. The discriminator model must then predict whether the given image is real or fake, just the same as CGAN does, as well as the data sample’s class label. Fig. 2 shows the difference in the CGAN and ACGAN architecture.

The discriminator and auxiliary classifier are independent networks in theory, but in fact, a single NN performs both tasks and produces two outputs. The first output is a single probability obtained from the sigmoid activation function, which denotes the input image’s realness and is optimized using binary cross-entropy, just like a standard GAN discriminator model. Like any other multi-class classification neural network model, the second output is a probability of the image belonging to each class obtained using the softmax activation function, optimized using categorical cross-entropy.

In ACGAN, the discriminator gives not only a probability distribution over the real source, but also a probability distribution over the relation labels. The loss function of the ACGAN is divided into two parts : (i) The log likelihood of the correct source — Source-loss (L_S) as in Equation 2 (ii) The log likelihood of the correct class — Class-loss (L_C) as in Equation 3.

The generator and discriminator compete over this loss function. The class-loss is maximized by both the generator and the discriminator. The source-loss, on the other hand, is a min-max problem. The generator tries to fool the discriminator by minimizing the source-loss. On the other hand, the discriminator aims at maximizing source-loss while preventing the generator from acquiring an advantage.

Given the relevance of insider threat analysis, we develop a generative model for data augmentation based on ACGAN in which the insider threat scenario labels y, as well as the noise sample z, are incorporated into the minmax optimization process. This study proposes SPCAGAN, a variation of the auxiliary classifier GAN by incorporating the linear manifold learning to maximize the similarity of real and fake data.

In this work, we use the manifold learning with respect to real and fake data. Manifold learning is about a class of algorithms to describe that low-dimensional, smooth structure of your high-dimensional data. Depending on the number of dimensions involved, a linear manifold can be considered a line, a plane, or a hyperplane. Data is usually represented in a high-dimensional space, and it is generally expected that a lower-dimensional linear manifold can effectively summarize the relationship between the variables. Most statistical theories and applications dealing with dimensionality reduction focus on linear dimensionality reduction and, by extension, linear manifold learning.

A linear manifold is a simple linear approximation to a more complex form of nonlinear manifold that can probably fit the data better. In both cases, the linear manifold’s intrinsic dimensionality is assumed to be significantly lower than the dimensionality of the data. The suggested method for achieving linear dimensionality reduction is to construct a smaller set of linear transformations of the input variables. Linear transformations are projection methods; therefore, the aim is to construct a sequence of low-dimensional projections of the input data with optimal qualities.

The challenge relies first in creating a good representation, which usually converts a high dimensional space, e.g. images, into a lower dimensional representation. Once we have such representation, we would need a technique to map new data samples into the representation, which we will refer as mapping to the latent space. Finally, a metric is applied in the latent space to determine the closeness of unseen data to the learned manifold insider scenarios. The proposed SPCAGAN architecture is given in the Fig. 3.

As seen in the Fig. 3, the generator loss is updated with the PCA similarity factor of the real and fake data samples to assure the distribution matching. We propose SPCAGAN which uses a loss function for regularization based on the PCA similarity factor [37] between real x and fake $\widehat{x}$ as in Equation 4.

This loss is minimized to assure the model regularization for data distribution matching. We estimate SPCA using a batch-based approach: for one batch during training, we use the GAN generated samples as the Z for that batch to compute the SPCA score for the original sample in the batch. In other words, the SPCA(real, fake) is defined efficiently across a set of original and synthetic samples. The algorithm for SPCAGAN training is given in Algorithm 1.

We intend to modify the training process in such a way that the GAN training takes into consideration the manifolds in the data so that the similarity of the data an be maximized. The discriminator is modified in such a way that the auxiliary classifier still works on the classification whereas the data similarity is computed using SPCA. Unlike previous augmentation approaches that use random transformations, SPCAGAN augmentation assures that the network’s training area is not restricted to the direct vicinity of a training sample.

Network Architecture: The number of layers used in generator and discriminator are fixed to five. The number of nodes in generator at each hidden layer is 32, 64, 64, 128, 512, 1024, the discriminator uses the same number of hidden nodes but in a descending order of size. The two neural networks were designed using dense layers. In generator, each layer’s activation function was Leaky ReLU, with the exception of the output layer, which used linear activation. The discriminator also uses Leaky ReLU at each dense layer. The output contains two layers : a sigmoid and a sparse categorical cross entropy. The proposed loss function is used in the SPCAGAN model while updating the generator. Once the training is complete, the generator model can be used to create any number of synthetic data for any required class. The next section explains how to use the SPCAGAN generated data for anomaly detection.

This section discusses our proposed hybrid model, which integrates deterministic and probabilistic methods to perform anomaly detection. Insider threat analysis using anomaly detection using ML/DL algorithms is a well-established method. Existing methods mainly focused on supervised learning with two classes: malicious and non-malicious. The scarcity of insider threat datasets and the diversity of malicious actions are two main reasons for binary classification-based analysis. Deep learning algorithms require a large number of samples in their training dataset to learn the representations and build a robust classifier, but tree-based models are known to work well on minimal data. Despite techniques such as XGBoost that perform well on unbalanced data, the need for more complicated learning models such as artificial neural networks continues to grow. We used a combination of ensemble methods and artificial neural networks, which have recently gained prominence.

Unlike existing methods, the proposed method considers the pre-defined scenario-based malicious activities and gives a deeper insight into the employee activities. Anomaly detection considers the different scenarios under malicious class and normal activities as non-malicious instances and performs the multi-class classification. The classifier tries to discriminate the anomalous activities from the other samples.

Probability distributions are used to express all uncertain elements in a model, including structural, parametric, out-of-distribution, and noise-related variables and their relationships to data. Deep neural networks with probabilistic layers that can express and process uncertainty are known as probabilistic neural networks. Bayesian Neural Networks (BNNs) are deep neural networks that control overfitting using posterior inference. The key concern with Bayesian neural networks is that their architecture makes computing for uncertainty in many subsequent layers unnecessary and expensive.

The recommendation in [40] is to utilize a few probabilistic layers at the network’s conclusion. The learning technique can be significantly simplified by training only a few probabilistic layers, and it solves numerous uncertainty-related issues like design and training while still producing valuable results from a Bayesian neural network perspective. It can be regarded as learning a hybrid BNN that includes parts of a standard neural network and shallow BNN. We designed a hybrid model that merges multiple deep learning models as feature extractors, followed by Bayesian network classification.

Gal and Ghahramani [41] recently proposed employing Dropout [42] at test time to evaluate predictive uncertainty using Monte Carlo dropout (MC-dropout). The ease of implementation with MC-dropout has led to its widespread use in practice. Dropout can also be understood as an ensemble model configuration that averages predictions among many NNs. Any reasonable approximation to the true Bayesian posterior distribution must depend on the training data. Hence, the ensemble interpretation sounds more realistic, especially when the training data does not affect dropout rates. This view motivates the usage of ensembles as an alternative method for evaluating predictive uncertainty.

A feasible solution is to employ hybrid Bayesian neural networks, which use a few probabilistic layers judiciously positioned in the networks. Our research uses a hybrid technique Bayesian Neural Networks, which takes advantage of the strengths of Bayesian inference in Artificial Neural Networks. Different classifiers are used in hybrid approaches, and their predictions are combined to train a meta-learning model. The hybrid system helps to improve the performance of a specific system.

We chose MLP and 1DCNN as the neural network algorithms for our experiment. MLP, a popular neural network, is suitable for numerical data and known for better generalization. In this work, we propose a hybrid BNN with (D + P) layers where there are D deterministic layers and P probabilistic layers. Fig. 4 provides the proposed hybrid learning model for anomaly detection.

The deterministic layers are used for learning task-specific representations, whereas the probabilistic layers are intended for prediction formulation and uncertainty assessment using the task-specific representation. The separation of representation learning and prediction generation is common in typical deep neural networks that use the last few dense layers for prediction generation. It is natural and straightforward to convert these typical deep neural networks to hybrid BNNs by just changing the last few dense layers to probabilistic dense layers.

Bayesian approaches are linked to stochastic learning algorithms in deep learning, and this connection is used to approximate the posterior in complex models. Dropout is a very effective stochastic regularization method for DNNs. The proposed model’s MLP and 1DCNN layers conduct feature extraction, while dense layers are employed to flatten the extracted features by the kernels in all layers. The Softmax layer computes categorical probabilities for organizing the final output. Softmax is used in the multiclassification technique to transform the output of multiple neurons to the (0, 1) classification interval. We carried out experiments to demonstrate the efficacy of the proposed approach.

Recent advances in generative modeling have highlighted the need for adequate quantitative and qualitative techniques to evaluate trainable models. Not only can reliable assessment measures aid in the evaluation of GAN models, but they also aid in discovering any inaccuracies in the data they generate. When people struggle to determine the quality of synthetic data, such as medical imaging, the requirement for acceptable measures becomes critical. It is not easy to assess a GAN model because different measures can yield different outcomes, and a high score on one evaluation criterion does not guarantee a high score on another [43].

As discussed in [44], the performance of a generative model is represented as a three-dimensional point, with each dimension corresponding to different model characteristics, namely fidelity, diversity, and generalization. Fidelity refers to the quality of a model’s synthetic samples, while diversity indicates how effectively these samples represent the extensive range of real samples, and generalization indicates how well a model overfits the training data.

Despite the measures like Inception Score (IS), Frechet Inception Distance (FID) FD, etc available for image data, the tabular data analysis is still an open research direction. In this work, we use multiple measures like statistical, clustering-based and manifold learning based metrics to analyze the fidelity. We use a Similarity Metric proposed by [45] which is an aggregation for the most prominent measures that describe the multivariate data distribution. The Silhouette coefficient [46] measures cluster cohesion and separation that quantifies how well a data point fits into its allocated cluster based on two factors: how near it is to other data points in the cluster and how far it is from points in other clusters. We compare real and synthetic data using the Silhouette score, which reveals information about mode collapse. Moreover, we exploit the SPCA [37] to compare the data similarity.

Diversity of data is presented using visualization of the data generated from synthetic models. We use the visual methods from t-SNE and kernel density estimation (KDE) to show the similarity of the original and synthetic data. Machine Learning Efficiency has been popularly used as a generalization metric for generative models. The generalization is tested using the synthetic data on various deep learning models for anomaly detection. Moreover, the robustness analysis also provides the efficiency of synthetic data on various other data poisoning attacks and deep learning models. The next section gives the detailed validation of the proposed approach.

Adversarial robustness [47], [48] refers to a model’s performance on test data when subjected to adversarial attacks, i.e., adversarial test classification accuracy. We expect to know the model’s performance in the worst-case scenario; thus, the adversary should be capable of launching attacks in white-box scenarios, in which the opponent has complete information of the model, including architectures, parameters, training data, and so on. Continuously inputting various types of adversarial samples and performing adversarial training improves the robustness of a deep network.

Machine learning algorithms accept numerical feature vectors as input. An adversarial attack is when you design input in a certain way to elicit the wrong predictions from the model. Researchers have found that adversarial examples transfer effectively between models, meaning that they can be built for a model but be effective against any other model trained on the same dataset. This is the transferability property of adversarial samples, which attackers can use when they don’t have access to all the model’s details.

We investigate adversarial attacks in a white box situation, in which the adversary has complete knowledge of the training process – specifically, the type of classifier employed, the training data, and any settings – but cannot influence training in any way. We use one-step attacks to examine how adversarially trained models perform. The adversarial samples are created from the distribution of test data and fed into the classifiers. The labels of the malicious samples have been reversed. Furthermore, if the learning model is used in an adversarial environment, it must be trained using adversarial approaches. Using the test split from the original data, we construct a target MLP model. The malicious instances are used to train the target classifier. The malicious instances of the test split are used to train the target classifier, ensuring that there are samples from all malicious scenarios. This target model is used to generate adversarial samples of malicious scenarios during validation. These samples are given random labels and injected into the test data with the original data. Specifically, the test data is trained by malicious samples with reversed labels. Here, we use white-box attacks like Fast Gradient Sign Method (FGSM) [49], and DeepFool [50] to generate the adversarial samples and study the impact on classifiers and provide the results in Section IV-D.

There is a scarcity of real-world data for insider threat detection. Usual practice is to use either data collected from real user data, or use synthetically generated data. Malicious insiders are predominantly employees in an organization with access rights. Data collection involves direct tracking and monitoring of the user actions and behaviors of the employees. This creates privacy and confidentiality concerns in an organization.

This work uses the CERT insider threat datasets, which are publicly available datasets used for research, development, and testing of insider threat mitigation strategies. We primarily choose dataset releases 4.2 (CERT r4.2) and 5.2 (CERT r5.2), which model organizations with 1000 and 2000 employees over 18 months, respectively. CERT datasets for the data collecting process (Section 4) include user activity logs in the following categories: log on/off, email, web, file and thumb drive connect, organizational structure, and user information. A variety of models, including topic, behavior, and psychometric models, are used to provide data that is as similar to reality as possible. CERT r4.2 has three malicious insider threat scenarios (1, 2, and 3), while CERT r5.2 includes an additional scenario (4). Each malicious insider in the CERT data falls under one of four typical insider threat scenarios: data exfiltration (scenario 1), intellectual property theft (scenarios 2–4), and IT sabotage (scenarios 3–5). (scenario 3). Insider scenarios are described in detail in [51]. Table II represents the summary of activity logs used for pre-processing.

In this section, we discuss the metrics and measures used for the performance evaluation of GAN and the anomaly detection used for insider threat detection. As discussed in Section III-D, we provide the results for fidelity, diversity and generalization of the GAN data. The generalization of the GAN data is tested using the machine learning efficacy - how the GAN data performs in machine learning models. Deep learning models are used for the anomaly detection. Hence, generalization is validated based on the performance of the GAN data on these models. The classification models are validated using commonly used performance metrics for imbalanced data, such as precision (P), recall (R), and f-score (F).

Recall, also referred as Detection Rate (DR), is a measure of a classifier’s completeness. The higher the recall, the more cases the classifier covers. If the DR is larger, the classification performance of the model is better. It is best suggested to have high precision and recall, and these metrics alone do not always accurately reflect the actual performance of various models. The majority class is typically regarded as the negative class, and it is due to the assumption that interesting samples are uncommon and thus considered positive.

Even though previous works achieved high precision and recall, no work has performed a detailed analysis of learning reports. We validated the results against the number of false positives and false negatives and analyzed them based on precision and recall. Given the importance of the confusion matrix, also known as an error matrix, we have included two additional metrics for the experimental evaluation: Cohen’s Kappa (Kappa) and Mathews Correlation Coefficient (MCC).

Cohen’s Kappa, also known as the Kappa score, measures the degree of agreement between the true and predicted values. Cohen’s Kappa is always less than or equal to 1, and a value of 0 or less indicates that the classifier is ineffective. Kappa score between 0.81–1 indicates nearly perfect agreement; the score reaches its maximum for balanced data. The Cohen’s kappa is the classification accuracy normalized by the imbalance of the classes in the data.

Precision, recall, and F-score are asymmetric; their values change when the classes are switched. The positive class is the class of interest for precision and recall. The confusion matrix’s True-Positive (TP), False-Positive (FP), and False-Negative (FN) values are used to compute them. These metrics do not use True-Negative (TN). Changes in TN never reflect changes in precision and recall. As a result, we consider MCC, a symmetric metric with a value between -1 and +1 inclusive. MCC considers all four values in the confusion matrix; no class is more important than the other, so switching between the negative and positive classes yields the same value. A value close to 1 indicates that both classes are accurately predicted. The experimental results are detailed in the Section IV-C.

In this section, we explain the performance of multi-class anomaly detection approach using deep learning algorithms for insider detection. Comprehensive set of experiments were conducted on original data as well as applying various data augmentation methods as follows : (i) on original skewed data with no data augmentation (ii) on CGAN augmented training data (iii) on CWGAN-GP augmented training data (iv) on ACGAN augmented training data and (v) SPCAGAN generated synthetic data in the training set. We used the algorithms MLP, 1DCNN and Bayesian Neural Network individually, ensemble of MLP and 1DCNN and a hybrid model using MLP and 1DCNN ensemble on BNN. These experiments helped in analyzing the behavior of data in various model setting.

ML/DL methods were used for building anomaly detection models on all the above mentioned data settings. We use text highlighted in bold to depict the interesting results in the tables with experimental results. The tables use the abbreviations Precision (P), Recall (R), F-score (F), Cohen’s Kappa (K) and Mathews Correlation Coefficient (MCC). Table III gives the performance of anomaly detection using multi-class classification.

The deep learning algorithms need more data to learn from the deep representative features. Hence, MLP and 1DCNN show remarkable improvement in the metrics. The performance of SPCAGAN augmented data for the anomaly detection is evident from the striking improvement in the metrics as shown in Table III. The results offer compelling evidence for supporting the efficiency of our proposed method.

In this section, we analyze the behavior of trained models in the presence of adversarial samples. The robustness of adversarial training is evaluated by creating samples using adversarial data generation methods like FGSM and DeepFool. The model trained using GAN augmented training data should be efficient when tested with FGSM and DeepFool generated adversarial samples. The already trained model is tested with these examples. FGSM [49] combines a white-box approach with a misclassification goal, and it tricks a neural network model into making wrong predictions.

We perform the experiments for the robustness to the white-box attacks in the following setting. Table V in Section IV-C provided the performance analysis for the various linear and non-linear classification models in the adversarial trained data. These models are validated against the adversarial examples generated using FGSM and DeepFool attacks and the results are given in Table IV.

We observe that models built using original and SPCAGAN augmented data are more resistant to FGSM one-step attack. We have observed this from the experiments with varying numbers of generated samples. The experiments show a significant drop in the performance metrics with DF attacks. The model with SPCAGAN augmentation applied on hybrid BNN yields better robustness on both datasets in deep learning models. The following section shows the effectiveness of SPCAGAN-based synthetic data compared to the other data generation methods.

There are various data generation methods available. In this section, we discuss some of the widely used data generation approaches like Random Over Sampling (ROS) [52], Synthetic Minority Oversampling Technique (SMOTE) [53], adding random noise to the data points [54], using Gaussian Mixture Models (GMM) [55]. We compare the performance of these techniques against the proposed method. To validate the proposed contributions, we perform the experimental analysis of the proposed approach and the comparison against other widely used augmentation techniques.

Oversampling refers to creating data samples from the minority classes to balance the data. Random Oversampling (ROS) and Synthetic Minority Oversampling Technique (SMOTE) are two popularly used oversampling approaches. Random Oversampling (ROS) is a data augmentation approach that selects random samples from the minority class with replacement and augments the training data with various copies of these instances so that a single instance can be selected multiple times. Random oversampling may enhance the likelihood of overfitting because it simply replicates minority class samples to increase sample count. A symbolic classifier, for example, may generate rules that appear to be accurate but only consider samples generated from one class.

SMOTE (Synthetic Minority Oversampling Technique) works by selecting a point at random from the minority class and computing its k-nearest neighbors. The synthetic points are added between the chosen point and its neighbors. The major limitation of using oversampling methods is that it increase the chances of overfitting as they replicate minority class events. The issue of oversampling is evident from the performance metrics given in the experimental results.

Gaussian mixture models (GMMs) are based on the notion that a finite number of Gaussian distributions exist, each of which denotes a cluster. As a result, a Gaussian Mixture Model tends to group data points from a single distribution. Gaussian Mixture Models are probabilistic models that use soft clustering to distribute points into multiple clusters. This method includes modeling data with a multivariate Gaussian distribution; the parameters are then computed using Maximum Likelihood Estimation (MLE), which is based on the mean and covariance matrix of the training data.

Table V provides the performance analysis of using various other methods for data augmentation. Overall better results are highlighted. Compared to Table III, the results in Table V show that no method provided agreeable performance for insider threat detection. GMM could perform better when compared to other methods on v4.2. But for v5.2, where the class imbalance is extreme, GMMs failed to generate quality data and hence poor performance in learning models.In the following section, we provide the authenticity evaluation of synthetic data generated using the proposed SPCAGAN.

This section deals with the authenticity of the GAN generated synthetic data. As discussed in Section III-D, we validated the synthetic data from three criteria : Fidelity, Diversity and the Generalization. We consider three metrics - SPCA, Similarity Score (SS) and Silhouette Coefficient (SC) for measuring the fidelity. Diversity is validated using visual methods like PCA and t-SNE. For generalization, we used the machine learning efficiency and is discussed in Section IV-C.

The SPCAGAN training is monitored in frequent intervals to study how the SPCA values progress. PCA is performed on both the datasets to find the principal components needed for maximum information utilization. In general, the number of principal components (PC) created will be the minimum value found of either (n-1) where n is the number of data points in a dataset or (p) where p is the number of variables in your data. We used the elbow method, a common heuristic in mathematical optimization, to get the number of principal components to be used in the SPCA calculation. CERT v4.2 requires three PCs whereas CERT v5.2 requires four. SPCA intuitively measures the similarity of two matrices by computing the squared cosine values of all the combinations of the first k principal components of two matrices. Fig. 5 shows the SPCA values changing during the GAN training for CERT v4.2 and CERT v5.2.

Table VI compares the fidelity measures for SPCA, similarity metric and the Silhouette score for the similarity between real and synthetic data from various data generation approaches.

From the table, it is evident that the SPCAGAN provided the best similarity between the original and the synthetic data. We considered all the three metrics for comparison. The compared methods include popular Random oversampling, SMOTE, Random noise addition, and other popular GAN methods like CGAN, CWGAN-GP and ACGAN.

Diversity of the data refers to the ability of the GAN model to generate synthetic data without mode collapse. Metrics for evaluating the similarity of synthetic data to original data distributions for numerical data types are not well developed. In the case of images, metrics such as Inception Score are used to compare synthetic and real images. Due to the lack of a proven metric for comparing the numerical data similarity between the original and synthetic data distributions, we utilize a visual approach based on kernel density estimates (KDE), Principal Component Analysis (PCA), and t-distributed Stochastic Neighbor Embedding (t-SNE) on the original and synthetic data.

To demonstrate the comparability of data distributions, we employed kernel density estimation, a non-parametric approach, on both the original and synthetic data. Due to space constraints, we use the visualization to illustrate two features. Fig. 6 depicts the kernel density estimation plot on two features L1 and L5.

PCA is a technique that converts n-dimensions of data into k-dimensions while maintaining as much information from the original dataset. PCA differs from t-SNE where the later one preserves the local neighbors of the data points. t-SNE can be considered as a manifold learning where the geometric data properties are used. t-SNE helps to increase the interpretability of data in the lower dimensions.

Figures 7 and 8 show the PCA and t-SNE visualization of the data on 2D for CERT v4.2 and CERT v5.2. PCA shows the dimension reduced data. Since it is a 2D visualization, the separate clusters are not visible; hence the overlap. In the t-SNE based visualization, the three clusters are the minority insider activity samples generated using the SPCAGAN model. CERT v4.2 displays three malicious scenarios in Fig.7 whereas CERT v5.2 shows four scenarios in Fig.8. The larger data spread indicates that there is no obvious mode-collapse.

Even though the literature shows numerous solution approaches being used in insider threat attacks, comparing these methods is challenging due to differences in many aspects like the data and the feature space used for training and validation and the learning methods applied for threat analysis. We tried to incorporate different studies where insider detection was studied previously in Section II-A. Many papers used the popular ensemble XGBoost for the insider threat analysis. In this work, the main aim was to use deep learning based models. The results from the proposed approach shows significant improvement than other works. Overall, the proposed SPCAGAN based adversarial training and hybrid BNN for anomaly detection yielded expected results in the experiments. In the next section, we summarize the work by discussing the conclusion and scope for future enhancements.

.