GraphFC: Customs Fraud Detection with Label Scarcity

There are variety of customs fraud, but undervaluation is the most common type. Importers or exporters declare the value of their trade goods lower than the actual value, mainly to avoid ad valorem customs duties and taxes (Rodrigue et al., 2016). For this work, we limit the definition of an illicit customs transaction to that of undervaluation. Given the volume of transactions and limited resources to inspect them, the customs administration prioritizes items that entail a maximum penalty. In this light, we formulate the problem of customs selection as follows:

By selecting two predicted values, $\hat{y}_{i}^{cls}$ and $\hat{y}_{i}^{rev}$ in descending order from all transactions $O=\{o_{1},\ldots,o_{n}\}$ , customs administration could identify the most suspicious transactions $O_{F}\subset O$ to be inspected. Meanwhile, since we focus on mitigating the low inspection ratio problem in this work, we assume we have the labeled set $L=\{(x_{i},y_{i}^{cls},y_{i}^{rev}),\forall i=1,\ldots,n\}$ and unlabeled set $U=\{(x_{i}),\forall i=n+1,\ldots,m\}$, where $|L|\ll|U|$.

Note that we also exhibit the model performance in the supervised setting and leave the results in Appendix, although a very high inspection rate( $>$ 90%) is a rarity for customs administrations.

One of the key insights of this work is to represent the customs transactions and their interactions as a graph and use GNNs to learn transaction-level embeddings by aggregating the neighborhood information. Traditional fraud detection methods (Kim et al., 2020; Vanhoeyveld et al., 2019; Filho and Wainer, 2008) judge the illicitness of a transaction solely depending on its features $x_{i}$ while the information of similar transactions is not considered.

We set up links (edges) between transactions (nodes) to capture any implicit patterns they might exhibit. These links could either be based on common categorical features, binned numerical features, or any other pre-defined rule. However, with even a relatively smaller customs office running into millions of transactions, we immediately run into a critical issue to exploding space complexity. A homogeneous graph formed by above rule suffers from a complexity of $O(n^{2})$ due to the pairwise relational nature, where $n$ denotes the number of transactions. To handle this issue, we introduce a set of virtual nodes $\mathcal{V_{C}}$ that are initially represented by the uniquely identifiable values/features (such as the importer id). Each node in $\mathcal{V_{C}}$ will only connect to transactions that share that feature value with it, and thus bringing down the space complexity from $O(n^{2})$ to $O(n)$.

Based on previous works like DATE (Kim et al., 2020), we choose importer ID and HS-code to be the categorical features defining the links between transactions. Thus, in this work, $\mathcal{V_{C}}$ is composed of the importer ID and HS-code with the uniquely identifiable values as their node IDs, as illustrated in the left half of Fig. 3. In line with this, when a prediction is made for a transaction $\bf{txn_{1}}$, the features of other transactions that share the same importer ID($\bf{txn_{2}}$) and HS-code($\bf{txn_{2}}$) would be propagated to $\bf{txn_{1}}$ to form a new representation, as demonstrated in the right side of Fig. 3. The detailed propagation rule will be described in Sec. 4. The transaction nodes $\mathcal{V_{T}}$ consist of both labeled and unlabeled transactions, which benefits the aggregation of additional information under a self- and semi-supervised setting.

The transaction graph could be represented as $\mathcal{G=(V,E)}$ where $\mathcal{V=V_{T}\cup V_{C}}$ is the node-set that denotes the combined individual transactions and the virtual nodes, and $\mathcal{E}$ denotes the connections between them based on the shared feature values. Let $\mathbf{D}\in\mathbb{R}^{h\times m}$ be the feature matrix that consists of $\mathbf{h=|\mathcal{V}|}$ nodes with their associated $\mathbf{m}$ features. It is worth noting that in the testing phase, only the historical transactions are considered for the inferring the transaction embedding. Also, the cross features from GBDT step become the features of nodes in $\mathcal{V_{T}}$, and the features of nodes in $\mathcal{V_{C}}$ are initialized with zero vectors $\vec{0}\in\mathbb{R}^{m}$.

Tree-based methodologies are a popular choice for building machine learning models on tabular data. Tabular data is known to have hyperplane-like decision boundaries, and tree-based models allow for hyperplane-based recursive partitioning of data space, such that points in the partitioned space share the same class. In this work, we follow the feature extraction method using GBDTs as (Kim et al., 2020). Gradient Boosting Decision Trees (GBDTs) methodology can be understood as an ensemble technique where multiple weak learner models are combined to provide a powerful overall learner. The decision path in the fitted trees could be seen as a new set of features, with each path representing a new cross feature made from the original features. Assume $C$ and $E_{T}$ respectively represent internal nodes and their connecting paths in a decision tree $D_{T}$. $C$ further can be viewed as consisting of three sets of nodes, the root node $\{c_{R}\}$, internal nodes $C_{I}$, and the leaf nodes $C_{E}$, where $C=\{c_{R}\}\cup C_{I}\cup C_{E}$. Nodes in $c_{I}\in C_{I}$ splits features into some decision space. After passing through internal nodes, an input feature vector $\mathbf{x}\in\mathbf{X}$ gets assigned to a leaf node $l_{e}$. Then, a decision rule is the path connecting the root node to the leaf node. For instance, “gross.weight >100kg $\land$ quantity < 5” is a two-node decision path or a cross-feature comprising of features gross.weight and quantity.

Inspired by some recent studies such as (Wang et al., 2018; He et al., 2014), we utilize state-of-the-art XGBoost model (Chen and Guestrin, 2016) and extract cross features from the fitted model.

Let us assume $W$ represents the total number of leaves in the ensemble. An input vector $\mathbf{x}$ ends up in one of the leaf nodes according to the decision rules of each fitted decision tree. Each activated leaf node in the $t$-th decision tree can be represented as a one-hot encoding vector $\mathcal{F}_{t}$. We concatenate these together and produce a multi-hot vector $\mathbf{p}\in\mathbb{R}^{W}$, where $1$ indicates the activated leaves and $0$, the non-activated ones. After the extraction of cross features, we apply transformation for all data points to obtain a new feature matrix $\mathbf{M}\in\mathbb{R}^{n\times W}$.

The strengths of learning latent representations of transactions via graph neural networks are mainly threefold. First, given a transaction node⁴⁴4In context of embeddings, we use the terms “node(s)” and “transaction(s)” invariably hereafter. $v_{t}\in\mathcal{V_{T}}$ and it’s representation vector $m^{k}_{t}$ at $k^{th}$ layer, graph neural network updates $m^{k}_{t}$ with the feature vector of adjacent nodes (i.e., $m_{j}^{k}~{}\forall j\in N(v_{t})$). Hence the nearby transactions in $\mathcal{G}$ could be included to form the representation for $v_{t}$. Second, the aggregation process takes the neighboring labeled and unlabeled transactions into account, a critical step in semi-supervised settings. Lastly, existing work (Kim et al., 2020) fails to extract informative patterns if either a new importer or an unseen item made the transaction. On the contrary, the inductive GNN generalizes to unseen nodes by exploring the node neighborhood. We elaborate the detailed framework in the following paragraphs.

In this work, GraphFC adopts GNN architecture inspired by GAT (Veličković et al., 2017), that has shown strong representation learning abilities among different graph learning tasks. Thanks to the flexible nature of our proposed approach, the backbone GNN architecture could be replaced with any GNN model. We demonstrate this by experimenting with GraphSAGE (Hamilton et al., 2017b) and RGCN (Schlichtkrull et al., 2018) (refer Table 2).

We initialize the node embedding for $\mathcal{V_{C}}$ with zero vectors then update it via the propagation from neighboring transaction features.

Unsupervised pretraining approaches in GNNs enable learning of hidden relational patterns and structural properties of the underlying network. The fact that such information can be learnt solely from observational data like node and edge features, without the need for ground-truth labels, lends an amazing flexibility to the representational learning of the GNNs (Bojchevski and Günnemann, 2018; Veličković et al., 2018; Kipf and Welling, 2016). Inspired by this, we adopt a self-supervised GNN pretraining scheme that learns transactions embeddings by using the given features of both the labeled and unlabeled transactions data.. Thereafter, we utilize these embeddings as a prior for the semi-supervised learning stage of GraphFC. The objective of the self-supervised step is to learn the embeddings of a node such that its representation in the latent space is closer to its neighboring nodes. By forcing the embedding of nearby nodes closer, GNN learns to preserve the graph structure level information as discussed in (Grover and Leskovec, 2016). Besides, it improves the generalization ability by aligning the distribution of unlabeled data closer to labeled data and thus reduce the extrapolation phenomenon.

We utilize the following self-supervised pretraining objective function (Hamilton et al., 2017b):

where $v$ is the immediate neighbor of node $u$, $P_{n}$ denotes the negative sampling distribution that gives a set of negative sampled nodes, and $R$ is the number of negative samples. By minimizing the objective function in Eq. 6, GraphFC is forced to lower the distance of representation of nearby nodes while pushing away the negative sampled nodes. Note that the pretraining operation is applied for all nodes in $\mathcal{V}$ which implies that both labeled and unlabeled information would be used to learn robust GNN parameters. Lastly, the trained parameters are saved and fine-tuned on the labeled dataset with Eq. 8.

To determine the illicitness of a given transaction, we propose to map the graph-based representation into scalars and train the neural network parameters via the following objective function. Inspired by (Kim et al., 2020), the dual-task learning framework is used to achieve two goals: 1. provide the probability of a transaction being illicit 2. predict the additional revenue (i.e., taxes) after inspecting suspicious transactions. Hence, we use the transaction representation (i.e., $s_{m}^{(k)}$) for both tasks of binary illicit classification and maximizing revenue prediction. Given the transaction feature $s_{m}^{(k)}$, we introduce the task-specific layer:

where $\textbf{r}_{1},\textbf{r}_{2}\in\mathbb{R}^{d}$ denotes the hidden vectors of task-specific layers that project $s_{m}^{(k)}$ into the prediction tasks of binary illicitness and raised revenue, respectively. $\phi$ is the sigmoid function. $\hat{y}^{cls}(s_{m}^{(k)})$ is the predicted probability of a transaction being illicit, and $\hat{y}^{rev}(s_{m}^{(k)})$ is the predicted raised revenue value of a transaction. The final objective function $\mathcal{L}_{\textsf{GraphFC}{}}$ is given by:

where $\Theta$ denotes all learnable model parameters, $\mathcal{L}_{cls}$(refer Eq. 9) is the cross-entropy loss for binary illicitness classification, $\mathcal{L}_{rev}$ (refer Eq. 9) is the mean-square loss for raised revenue prediction. The hyperparameter $\alpha$ is used to balance the contributions.

Finally, we use mini-batch gradient descent to optimize the objective function $\mathcal{L}_{\textsf{GraphFC}{}}$, along with the Ranger (Wright, 2019) optimizer.

We employed multi-year transaction-level import data from three partner countries, released by WCO for research usage. Because the data contains only the import transactions and those reported with undervaluation, we are only working with limited data from a larger dataset. Due to data confidentiality policies, we abbreviate these countries as A, B, and C. Subject to data availability, the three datasets have a different number of records, details of which are summarized in Table 1. Further, as data belongs to different customs administrations, it varies slightly in the features. Nonetheless, the most relevant features are the same across the datasets. Among others, these include the HS-code, importer ids, the notional value of the transaction, date, taxes paid, the quantity of the goods. We also include a binary feature representing historic fraudulent behavior or total taxes paid per unit value.

Because of our proposed design of introducing the virtual nodes for HS-codes and importer ids as presented in Sec 3.2, the number of edges in the resultant graph structures of each of the datasets would be twice the number of transactions in it.

We run an elaborate set of experiments that aim to answer the following: (a) Subject to the limited inspection rate by the customs officers, how well can GraphFC identify the fraudulent transactions? (b) Similarly, what additional revenue can GraphFC generate by inspection of items suggested by it? (c) How unlabeled data helps in semi-supervised learning? (d) Do different components in GraphFC contribute to making better predictions?, and finally, (e) Is the model robust to different backbone networks and inspection rates.

GraphFC performance is compared with 4 baseline methods:

• •

  XGBoost (Chen and Guestrin, 2016): XGBoost is a tree-based model which is widely used for modeling tabular data.

• •

  Tabnet (Arik and Pfister, 2019): Tabnet is a self-supervised learning-based framework especially designed for tabular data.

• •

  VIME (Yoon et al., 2020): VIME adopts self-and semi-supervised learning techniques for tabular data modeling.

• •

  DATE (Kim et al., 2020): This fraud detection method which utilizes tree-aware embeddings and attention network.

• •

  $\textsf{GraphFC}{}_{\text{SAGE}}$: A GraphFC variant using GraphSAGE (Hamilton et al., 2017b) aggregator in message passing.

• •

  $\textsf{GraphFC}{}_{\text{RGCN}}$: A GraphFC variant using RGCN (Schlichtkrull et al., 2018) aggregator designed for relational and heterogeneous graphs.

• •

  GraphFC: Proposed model with GAT (Velickovic et al., 2018) based aggregator.

GraphFC improves its representation ability by utilizing the unlabeled data in the pretraining and fine-tuning stage. The key to making use of unlabeled data lies in the construction of transaction graph as presented in Fig. 3. The transaction graph comprises of $V_{T}$ and $V_{C}$, where $V_{T}$ includes both labeled and unlabeled transactions. To verify the effectiveness of unlabeled data in $V_{T}$, two variants of $\mathcal{G}$ are made with the following rule: (a) $\mathcal{G}_{L}$: keeps only the labeled transaction and its corresponding edges with $V_{C}$. (b) $\mathcal{G}_{U}$: keeps only the unlabeled transaction and its corresponding edges with $V_{C}$. We then compare the results using different graphs in the pretraining(Q) and fine-tuning (F) stage With 6 combinations and list the performance with a semi-supervised setting of B-Data in Table 3. Each row represents the graphs used in pretraining and fine-tuning, for example, $Q(\mathcal{G}_{L})+F(\mathcal{G}_{U})$ means pretraining with graph $\mathcal{G}_{L}$ and fine-tune on graph $\mathcal{G}_{U}$.

First, it is worth noticing that $Q(\mathcal{G}_{L})+F(\mathcal{G}_{L})$ is the only variant that solely uses labeled data without inclusion of any unlabeled data in the training process. The results show that it performs the worst among all test cases. The takeaway here is that the using unlabeled data in the pretraining or fine-tuning process leads to a significant improvement in the model performance. Second, we compare the result of fine-tuning on $\mathcal{G}_{L}$ and $\mathcal{G}$. The results show that fine-tuning with unlabeled data generally performs better, which again shows the advantage of jointly considering labeled and unlabeled information. Lastly, we observe the difference between $Q(\mathcal{G}_{L})$ , $Q(\mathcal{G}_{U})$ and $Q(\mathcal{G})$. $Q(\mathcal{G}_{L})$ is significantly worse than the others while $Q(\mathcal{G}_{U})$ and $Q(\mathcal{G})$ usually gives similar performance. This final result adds to the already evident effectiveness of adding unlabeled data in the pretraining stage. It not only provides more training instances for learning the weight matrices (i.e., $\mathbf{W}_{1}^{(k)},\mathbf{W}_{2}^{(k)}$) but makes the nearby transaction closer in embedding space, leading to better predictive capabilities.

To further explore the effect of including unlabeled data in GraphFC, we train two variants of GraphFC and mapped their transaction embedding obtainable before the output layer into 2-dimensional space using TSNE, and present the result in Figure LABEL:fig:tsne_plot. Figure LABEL:subfig:L_tsne presents the variant of GraphFC where we remove all of the unlabeled data in $\mathcal{G}$ (refer to the setting of $Q(\mathcal{G}_{L})+F(\mathcal{G}_{L})$), whereas for Figure LABEL:subfig:nopretrian_tsne, the pretraining step is ommited, and Figure LABEL:subfig:LU_tsne is the result of GraphFC. Figure LABEL:subfig:LU_tsne is the best performing case where legitimate and illicit transactions are separated well into two tightly clustered sets. In Figure LABEL:subfig:L_tsne, we can observe diverse patterns that are not clearly separable. Figure LABEL:subfig:nopretrian_tsne has better separation but data points are more dispersed than as compared to Figure LABEL:subfig:LU_tsne. To sum up, this analysis shows the importance of unlabeled data in learning high quality embedding vectors that played a key role for mitigating effects of label scarcity.

Inductive learning can be understood as the ability of the model to learn from the historic data and generalize it to new, unseen cases. Given the huge volume of transactions, the model will consistently come across input data that has unseen features, such as new items and importers. In the previous work DATE (Kim et al., 2020), the representations for every importer and HS-code were learnt, however, in a semi-supervised setting, such a learning becomes infeasible. GraphFC provides a novel solution for inductive learning by linking the newly observed importer and item to the target transaction and its representation could be updated via multiple messages passing operation. To measure the degree of the inductive phenomenon, we introduce Out-of-Sample-Ratio (OSR), which is defined as: $OSR=|P_{un}|/(|P_{s}|+|P_{un}|),$ where $|P_{s}|$ and $|P_{un}|$ denote the number of unique states (refer inductive setting in 3.1 for definition) observed and unobserved in the training data, respectively. The higher OSR is, the more unseen data states are present in the data. In this work, we focus on analyzing the behavior of importer and HS-code.

Inductive analysis in semi-supervised learning We evaluate the performance of inductive learning by selecting a subset of transactions from testing data where the importer ID or HS-code were not observed in training data. The result is presented in and presents the result in Table 4 and 5. We compare GraphFC with DATE as DATE being the state-of-the-art method for transductive learning. In Table 4, it demonstrates the performance on predicting transactions made by unseen importers. GraphFC outperforms DATE with a significant improvement as OSR increases. Especially in B-data, the OSR is 49.43% and GraphFC gains 7x improvement in terms of Pre@1%. On the other hand, Table 5 also shows the superiority against DATE on unseen HS-code. Additionally, in Figure 7 we also show the effect of varying OSR rates by controlling HS codes and importer id. GraphFC outperforms the baselines in all cases.

To establish the contribution of different components in GraphFC, we ablate its core components and evaluate its performance. Specifically, we evaluate the following settings:

• •

  GraphFC: Use full model.

• •

  No unsupervised pretraining (GNN_semi): Skip the unsupervised pre-train step.

• •

  Joint training (GNN_joint): Remove pretraining and jointly optimizing E.q 6 and Eq. 8.

• •

  No GBDT (GNN_only): Remove the GBDT step of the model and utilize the data from transactions directly.

• •

  No item/importer nodes (GNN_sparse): Remove one of the categorical nodes utilized for building the graph structure and build a sparser graph.

Results of different variants are demonstrate in Table 6. It can be noticed that removing any component in GraphFC leads to degradation of performance. The only exception is for GraphFC_semi that delivers similar performance in C-data. Among these variants, GraphFC_sparse and GraphFC_only perform poorly which indicates that both the GBDT step, and utilizing the importer and HS-code for constructing the transaction graph play an important role in GraphFC.

In general, neither GraphFC_semi nor GraphFC_joint improves GraphFC which verifies the necessity our pretraining step. Specifically, the difference between GraphFC and GraphFC_semi, GraphFC_joint are significant when $n$ is small (e.g. $n=1\%$) which is important since customs usually maintain a low inpsection rate lower than 5%.