The Devil is in the Conflict: Disentangled Information Graph Neural Networks For Fraud Detection

The core idea of graph-based fraud detection task is taking the advantages of GNNs to get the discriminative node embeddings, and find out the malicious ones in the latent space. Examples include [11, 15, 16] for review fraud detection, [2, 3] for fake news detection and [5, 6, 17, 18, 19] for financial fraud detection. Ma et al. [20] provides a comprehensive investigation on graph-based fraud detection.

Most of existing GNNs methods holds homophilic assumption that neighbor nodes share same labels or similar features. However, fraudsters will try to conceal themselves, so that their features are inconsistent with their camouflage behaviors. Some graph-based fraud detection works have noticed this problem. GraphConsis [11] pioneers to formulate and tackle the inconsistency problem. They introduce three kinds of inconsistency phenomenon existing in fraud networks. CARE-GNN [15] devises a label-aware similarity measure to find informative neighboring nodes and utilizes reinforcement learning to select similar neighbors. FRAUDRE [21] aggregates difference between adjacent node pairs. PC-GNN [6] devises a choose operation to select beneficial neighbors based on feature similarity. IHGAT [22] is devised to encode both sequence-like intentions and relationship among transactions for leveraging the cross-interaction information.

Our model is different from all above works. We innovatively disentangle topology and attribute and consider graph learning as a multi-view learning problem, instead of measuring similarity between adjacent node pairs.

Topology and attribute are two essential compositions of graphs. However existing state-of-the-art GNN models are disable to effectively fuse topological structure and node attributes. AM-GCN [14] uses $k$-nearest neighbor to construct feature graph and combine it with topological structure view and common embeddings. SCRL [23] designs a self-supervised approach to maximize the agreement of the embeddings in the topology graph and the feature graph. A recent work [13] claims that the interference between topology and attribute is mainly ascribed to compromises between them. LINKX [24] processes node attributes and topological structure in an orthogonal manner. In this paper, we also follow this idea and extend it by proposing a novel architecture and optimization objective.

Information-theoretic methods have been gaining momentum in recent years, which take into consideration the mutual dependency of different views. MIB[25] extends the information bottleneck principle to unsupervised multi-view setting to discard superfluous information. DVIB[26] and CMIB[27] leverage mutual information constrains to better preserve shared and private information of multi-view learning. To cope with intractable computation of mutual information, these methods adopt variational inference to optimize objective lower and upper bounds. In comparison, our model propose a novel optimization objective to reconcile consistency and complementarity between topology and attribute views. Equipped with variational inference, we also approximate the mutual information with derived bounds.

Definition 1. Graph-based Fraud Detection. Given a fraud network $\mathcal{G}=(\mathcal{V},\mathbf{A},\mathbf{X})$, where $\mathcal{V}=\{v_{1},v_{2},\ldots,v_{N}\}$ is the set of nodes; $\mathbf{A}\in\mathbb{R}^{N\times N}$ is the adjacency matrix, if $v_{i}$ and $v_{j}$ are connected, $\mathbf{A}_{ij}=1$, otherwise, $\mathbf{A}_{ij}=0$; $\mathbf{X}\in\mathbb{R}^{N\times D}$ denotes node feature matrix, each node $v_{i}$ is associated with a $D$-dimensional feature vector $\mathbf{x}_{i}$ and a label $y_{i}\in\{0,1\}$, where 0 denotes the node is a normal user (negative) and 1 indicates it is a fraudster (positive). The core idea of graph-based fraud detection is to learn discriminative node embeddings to detect the anomaly samples in latent space.

Definition 2. Graph Neural Networks. Most of GNNs follow message passing mechanism which uses feature transformation and aggregation operations to capture the structural and attribute information. One of the most popular and representative GNNs model is graph convolutional networks (GCNs). The forward inference at the $l$-th layer of GCNs is formally defined as:

where $\sigma(\cdot)$ is nonlinear activation function, $\mathbf{W}^{(l)}\in\mathbb{R}^{d\times d}$ is the linear transformation matrix, $\mathbf{H}^{(l)}$ denotes the node embedding matrix at the $l$-th layer, and $\mathbf{H}^{(0)}=\mathbf{X}$, $\hat{\mathbf{A}}$ is the normalized adjacency matrix, which can be implemented by row-normalization, $\hat{\mathbf{A}}=\mathbf{D}^{-1}(\mathbf{A}+\mathbf{I})$ or symmetric normalization, $\hat{\mathbf{A}}=\mathbf{D}^{-\frac{1}{2}}(\mathbf{A}+\mathbf{I})\mathbf{D}^{-\frac{1}{2}}$, and $\mathbf{D}$ is a diagonal matrix, $\mathbf{I}$ is an identity matrix.

Interestingly, some works [12] have declared that representative GNN models can be regarded as solving a Graph Signal Denoising problem, which given a noisy signal $\mathbf{S}\in\mathbb{R}^{N\times d_{in}}$ on graph $\mathcal{G}$, the goal is to recover a clean signal $\mathbf{F}\in\mathbb{R}^{N\times d_{out}}$:

where the first term guides output signal $\mathbf{F}$ similar to original signal $\mathbf{S}$ and the second term encourages signal smoothness on graph.

It is universally acknowledged that topology and attribute are of vital importance for graph learning. However, in graph fraud detection scenario, traditional message passing along neighboring nodes is inappropriate as graph signal smoothing makes fraudsters more indistinguishable. To alleviate the inconsistency problem, we disentangle the topology and attribute information and encode them in parallel.

Given an attributed fraud network $\mathcal{G}$, it can be disentangled into topology view $\mathbf{A}$ and attribute view $\mathbf{X}$. Here we provide two view encoders $f_{A},f_{X}$ for each input view, as shown in Figure 2. Specifically, we employ Multi-Layer Perceptron (MLP) as encoders to obtain view-specific embeddings $\mathbf{Z}^{A},\mathbf{Z}^{X}\in\mathbb{R}^{N\times d}$:

in which $d$ is the embedding dimension. With these two embeddings, we need to fuse them to obtain final representation and extract task-relevant information.

Now we have two view-specific embeddings $\mathbf{Z}^{A}$ and $\mathbf{Z}^{X}$, we then perform cross-view fusion by utilizing attention mechanism. The attention value $\omega_{i}$ can be represented as:

where $q$ denotes the learnable attention vector, $\mathbf{W}$ is the weight matrix and $b$ is bias vector. Thus, we can get the attention values $\omega^{A}_{i}$ and $\omega^{X}_{i}$ for view-specific embeddings $\mathbf{z}^{A}_{i}$ and $\mathbf{z}^{X}_{i}$, respectively. Then we normalize them via softmax function to get the final weight:

Larger attention weight $\alpha_{i}$ implies that the corresponding embeddings is more important, and it is determined by specific dataset. Then the final output embedding $\mathbf{z}_{i}$ can be combined by two view-specific embeddings with its corresponding attention weight as:

And we put it into a linear classifier, while training by a cross-entropy loss function:

in which $\mathbf{W}^{\prime}$ and $b^{\prime}$ is the weight matrix and bias vector of linear classifier, $\sigma$ is a softmax function, and $\mathcal{V}_{\text{train}}$ is the training node set.

Up to now, we have discussed how to get view-specific embeddings and fuse them with attention mechanism. However, as mentioned in [13], the representative GNN models tend to deteriorate their expressive power due to interference between attribute and topology. In spite of decoupling operation, it is still impractical to look forward to injecting mutual-exclusive learning ability to our model simply equipped with attention mechanism. In other words, we need to seek some principles to guide the training procedure. By leveraging the information theory, we propose a novel optimization objective to alleviate the aforementioned problem. Furthermore, we derive the variational bound of our optimization objective and discuss the intrinsic effect and intuitive insight. Without loss of generality, we let $\mathbf{X}_{1},\mathbf{X}_{2}$ to represent original views and $\mathbf{Z}_{1},\mathbf{Z}_{2}$ to represent view-specific embeddings for ease of reading.

To answer the RQ1, we analyze the attention values and visualize them for investigating whether the attention values learned by our model is meaningful. The attention changing trends are shown in Figure 4. The x-axis is the number of training epochs and y-axis is the average attention value. With the training epoch increasing, the difference between the corresponding attention values of topology and attribute begin to be striking. We can observe that DIGNN pays more attention on attribute and topology on YelpChi and Amazon datasets respectively. It demonstrates our model has a strong capability to extract the task-relevant information from these two views.

To answer the RQ2, we compare the performance of DIGNN with state-of-the-art methods. The corresponding F1-macro, AUC and GMean scores are shown in Table II, we have the following two observations.

First, DIGNN significantly boosts the performance for all metrics on YelpChi and Amazon datasets than other SOTA baseline methods. In YelpChi dataset, our model obtains 9.62%, 8.11%, and 5.28% improvement respectively in F1-macro, AUC and GMean. We can observe that PC-GNN outperforms other baselines in most metrics, but our model can still surpass it by a significant margin. In Amazon dataset, graph-based fraud detection methods have already achieved high performance and the increasing room is limited. But our model can still get improvements, with 3.14% improvement in F1-macro, 2.47% improvement in AUC and 3.29% improvement in GMean. In addition, the relatively low standard deviation of DIGNN shows that our model is stable.

Second, the compared baseline methods can be divided into two groups, traditional MP-GNNs and graph-based fraud detection methods. GCN, GAT, GraphSAGE are tradition GNN models, and DR-GCN is designed for imbalanced node classes. They do not consider the inconsistency problem so that we can observe these models get poor performance on YelpChi and Amazon datasets. Because Amazon dataset has a more skewed label distribution (fraudsters only occupy 6.87% of all samples), more intra-class edges which are beneficial to traditional MP-GNNs are appeared in graph. Thus these methods have a relatively satisfactory performance on Amazon. CARE-GNN and PC-GNN are graph-based fraud detection methods, they both sample neighbors according to similarity measure, which can alleviate inconsistency problem to a certain degree. Therefore, they can perform better on these two datasets. However, sampling neighbor strategy may discard a lot of information, and it can lead to sub-optimal results. Instead, our DIGNN model abandons this practice, and disentangles original graph into topological structure and node attributes, then processes them in parallel.

In general, DIGNN outperforms all baselines in F1-macro, AUC and GMean on YelpChi and Amazon datasets, which can demonstrate the effectiveness of our model.

To answer the RQ3, we compare DIGNN with two corresponding variants DIGNN_∖S and DIGNN_∖M. The results of two datasets are shown in Table II. We can observe that DIGNN surpasses its variants in most of metrics. For DIGNN_∖M, its overall performance on Yelpchi and Amazon is inferior to complete model, which verifies the effectiveness of our proposed mutual information objective. It is noting that the DIGNN_∖M is on-par with DIGNN on Amazon dataset evaluated by F1-macro. This could attribute to smaller fraud rate of Amazon, thus DIGNN_∖M pay more attention to majority class without the guidance of mutual information compared with DIGNN. For DIGNN_∖S, we can observe that DIGNN is evidently better than model without sampling strategy. We suppose it is caused by the noise information of the structure view. Sampling strategy play a denoising effect on structural information to some extent.

To answer RQ4, we further evaluate the performance of DIGNN with respect to the training ratio, hidden dimension $d$ and hyperparameters $\alpha,\beta$. For training ratio, we vary the percentage of training nodes from 10% to 50%, and compare DIGNN with other two baselines, CARE-GNN and PC-GNN. Figure 5 shows the performance of F1-macro, AUC and GMean on YelpChi dataset. We can observe that DIGNN always achieves best performance among the three models. When the training ratio is 10%, DIGNN still performs better than PC-GNN training on 50% samples. And DIGNN surpasses CARE-GNN and PC-GNN by a large margin in AUC.

For hidden dimension $d$, we study the performance of DIGNN with various hidden dimension number $d$ from 8 to 64. And the results are presented in Figure 6. With the increase of hidden dimension $d$, the performances improve first, but then start to grow slowly. It is relatively stable with respect to hidden dimension $d$.

For hyperparameters $\alpha$ and $\beta$, we vary these two hyperparameters from 0 to 1, and the corresponding results are shown in Figure 7. Considering the limit space, we only present AUC performance on YelpChi and Amazon datasets. It can be observed that the optimal selection of these two hyper-parameters varies greatly on the different datasets. In the YelpChi dataset, higher AUC performance can be achieved by selecting larger $\beta$ ($\beta\geq 0.8$). And in the Amazon dataset, larger $\alpha$ ($\alpha\geq 0.6$) and smaller $\beta$ ($\beta\leq 0.4$) can get a better result.

In order to show the effectiveness of different models more intuitively, we visualize the learned node embeddings on YelpChi dataset. Specifically, we compare our DIGNN model with the other three models. We select one traditional MP-GNN, GCN, and two graph-based fraud detection models, CARE-GNN and PC-GNN. Firstly, we use the 32-dimensional output embedding on the last layer of these models before softmax function. And then we employ the t-SNE [39] to map the 32-dimensional embedding into 2-dimensional space for visualization. Because of the imbalanced class distribution, we randomly sample the same number of benign samples as fraudster samples for better visibility. The results of YelpChi are showed in Figure 8, and orange dots represent fraudsters, blue dots represent benign entities.

We can observe that due to the strong inductive bias of homophilic, GCN is disable to learn discriminative node embeddings. For CARE-GNN and PC-GNN, although they alleviate the inconsistency problem, they still fail to seperate the embeddings of fraudsters from that of the benign entities. Conversely, DIGNN achieves inter-class separation obviously, the overlap of the two kinds of nodes is relatively small. Consequently, it can verify the effectiveness of our proposed DIGNN model.