Fake Reviewer Group Detection in Online Review Systems

Since Jindal and Liu[1] propose the fake review detection problem, fake reviews in online shopping websites have gained increasing concern. Most previous studies in this area can be categorized into the following types: behavior-based, linguistics-based, and relation-based. For behavior-based methods, Li et al.[2] consider semi-supervised models that utilize reviews’ and users’ views by co-training processes. Xie et al.[6] study the temporal pattern of users’ behavior to uncover fake reviewers. For linguistics-based methods, Feng et al.[3] exploit features generated from Context Free Grammar parse trees to make improvements. Neiterman et al.[7] develop a multilingual speech-based detection model. Considering graphs’ advantages in capturing inter-dependent properties from metadata, some proposed graph-based approaches. Wang et al.[14] present a heterogeneous graph model and use an iterative approach to discover the propagation in users. Li et al.[15] build a user-IP-review graph that connects reviews from the same users and IPs.

Most existing methods focus on fake reviewer detection at the individual level rather than the group level. However, most review spam activities are well-organized and thus fake reviewers form collusive groups, especially along with the rising of crowdsourcing/crowdturfing systems [11]. Fake reviewer groups are more powerful in distorting a product’s reputation. They are also much more difficult to discover as they may appear benign at the individual level. Mukherjee et al.[16] propose a frequent itemset mining method to detects fake reviewer groups. Wang et al.[8] generate loose fake reviewer groups from graphs in a divide and conquer manner. Wang et al.[9] later introduce a new approach that decompose the entire reviewer graph into small fake reviewer groups by a minimum cut algorithm. Dhawan et al.[17] propose DeFrauder that harnesses group indicators from behavior and graph features to discover and rank fake reviewer groups. However, they do not consider group-level and individual-level features in a unified manner, ignoring the effectiveness of traditional features in group detection.

With the resurgence in deep learning, researchers have begun to apply GNNs to graph-based anomaly detection[18]. Compared with traditional approaches, GNNs can effectively aggregate information from neighbors and extract features in graphs. Basic GNNs [19] complete aggregation via the mean function and degree of neighbors, after which many improved algorithms followed. GraphSAGE [20] merges messages of self-node features from previous layers and introduces the concept of sampling in GNNs to alleviate the cost problem in computing. GATs [21] use a pair-wise function on nodes that updates weights to learn node features.

Besides the application in fake news detection[22], financial assessment[23], many researchers also use GNNs for review spam problem. For example, Wang et al.[24] trains a GCN[19] model to find fraudsters. Li et al.[25] constructs two graphs and uses GCN to learn reviews’ local and global context. Dou et al. [26] strengthens the aggregation process in GNN to defend from camouflages. However, GNNs in fake review detection are mostly used for node representation and individual reviews/reviewers classification. This ignores the problem of fake reviewer groups and fails to recognize GNNs’ effectiveness in graph pooling especially graph clustering[27]. Moreover, Wang et al.[13] demonstrates that jointly performing representation and classification with GNNs would deteriorate training effectiveness if there is users’ behavior pattern and label semantics are not conformed, which is a common case in real-world settings.

Compared with previous models, REAL contributes comprehensively to the issues above. It uses GNNs as a building block and utilizes information from users’ behavior and graph structure collectively to discover the most suspicious groups.

We construct an attributed ”product-rating” graph $\mathcal{G}=\mathcal{(V,E)}$ that aims to incorporate more information from metadata. As demonstrated in Fig. 2, a node $v_{ij}\in\mathbf{V}$ represents a product $p_{i}$ and one kind of its ratings $r_{j}$, namely, a product-rating pair $(p_{i},r_{j})$. The node attribute $\mathbf{a}^{n}_{ij}$ is defined as the reviewer set where each member has given product $p_{i}$ the rating $r_{j}$. The edge $e_{(ij,mn)}$ indicates reviewer sets in which members share identical rating behavior on the two products $p_{i},p_{m}$, and the edge attribute $\mathbf{a^{e}_{(ij,mn)}}$ denotes the co-review and co-rating reviewer set $\mathcal{R}_{(ij,mn)}$. For example, if reviewer Alice and reviewer Bob both gave $p_{i}$ the score of $r_{j}$ and $p_{m}$ the score of $r_{n}$, then Alice and Bob should be included in the $\mathcal{R}_{(ij,mn)}$. Note that we assume that each user can only give a product one rating.

We first extract candidate groups based on cluster results before using group-level anomaly indicators. We handle this task by using methods in graph clustering, which is also commonly known as dividing networks into multiple communities. A community in networks is popularly defined as a group that has tighter relation within, which aligns with our description of fake reviewer groups.

Traditional approaches in graph clustering use cut-based metrics to detect communities in graphs[28]. However, the good cuts these methods assume[29] are vulnerable as one node may belong to more than one community. In our scenario, a product could be involved in several spam activities and a fake reviewer could participate in several collusive groups. Meanwhile, modularity[30] based metrics tackle this problem by maximizing intra-cluster relation and minimizing inter-cluster relation. Specifically, given each node $i$ its degree $d_{i}$, a random graph with $m$ edges is generated, and the node pair($u,v)$ is linked with probability $d_{u}d_{v}/2m$. Modularity is then calculated as follows:

where $c_{x}$ denotes the cluster a node is assigned to. Modularity calculates the divergence of the edges within a cluster from the expected one, and we harness it to characterize and detect overlapping collusive groups in $\mathcal{G}$.

To solve the NP-hard problem of maximizing the modularity, let $d$ be the degree vector, modularity matrix $\mathbf{C}\in 0,1^{n\times k}$ be the cluster assignment and $\mathbf{B}=\mathbf{A}-\frac{\mathbf{d}\mathbf{d}^{\top}}{2m}$, and modularity can be refined as

where Tr(*) denotes the trace of matrix *.

While this process utilizes completely the structure of graphs, we still face the cost problem when adapting it to our attributed graphs.

The aim of generalizing convolutions to graph domain is to encode the nodes with signals in the receptive fields. Given a graph $\mathcal{G}=(\mathcal{V,\mathcal{E}})$ with nodes count $N=|\mathcal{V}|$, where each node is represented with a feature vector. This way the convolution operator can be represented by Hadamard product in the Fourier domain.

For a GCN with $L$ convolutional layers, each layer embeds nodes by aggregating its neighbors from previous layer. Consider the common practice for a GCN:

where $\widetilde{\mathbf{A}}=\hat{\mathbf{D}}^{-\frac{1}{2}}\hat{\mathbf{A}}\hat{\mathbf{D}}^{-\frac{1}{2}}$, $\hat{\mathbf{A}}=\mathbf{A}+\mathbf{I}$, $\hat{\mathbf{D}}$ is the diagonal node degree matrix of $\hat{\mathbf{A}}$. $\mathbf{H}^{(0)}=\mathbf{X}$ is the input in the first layer. $\mathbf{X}$ is the feature matrix. $\mathbf{W}^{(t)}$ is a learnable matrix shared among all nodes at layer $t$, and $\sigma(\cdot)$ is the activation function.

In REAL, we use SeLU[31] for activation, which is proven to benefit convergence. Also, we skip the self-loop and insert a weight matrix $\mathbf{W}_{s}$ for skip connection.

Therefore, the $t$-th output layer for $\mathbf{H}^{(t+1)}$ is

To conduct graph clustering on $\mathcal{G}$ and select candidate collusive groups, we refer to deep modularity networks that use modularity-based loss function for optimization. In detail, the model gets the cluster assignment matrix via softmax on graph convolution networks. We obtain $\mathbf{C}$ as follows:

We then introduce spectral modularity in its optimization function. However, allocating every node to the same cluster may trigger the local minima problem which harms optimization[32]. Therefore we assign spectral clustering a regularization to guarantee the informativeness in clustering. To avoid being too restrictive when performed with softmax, the collapse regularization is relaxed as$\frac{\sqrt{k}}{n}\left\|\sum_{i}\mathbf{C}_{i}^{\top}\right\|_{F}-1$, where $\left\|\cdot\right\|_{F}$ is the Frobenius norm, $n$ is the number of nodes and $k$ is the number of the clusters.

So the complete loss function is defined as follows:

Our next step is to evaluate candidate groups with anomaly indicators and sort out the most suspicious ones. After graph clustering, nodes in the graph $\mathcal{G}$ are assigned to the clusters. We then intersect the reviewer sets of targeted products in the same clusters, which are also the attributes of nodes, and get the group members of each cluster. To grasp messages from reviewers’ behavior, content and underlying relation, we adopt the indicators discussed below. Note that according to Xu et al.[33], language features can be easily imitated by fake reviewers. Furthermore, many online review systems only allow rating for products without comments in texts. Therefore, in our settings, we focus on rating behaviors and users’ temporal characteristics, discarding the language features.

Given a candidate group $g$ and its members $r_{i}\in\mathcal{R}(g)$, we measure its suspiciousness by ranking the anomaly score $\Omega$. To be specific, after calculating its scores of Group Anomaly Compactness $\Pi$ and the individual-level indicators, each score is scaled between 0 and 1 with min-max Normalization. Then $g$’s anomaly score $\Omega$ is calculated as follows:

where Group Anomaly Compactness multiples 3 before added to the final anomaly scores as we emphasize more on group-level anomaly messages.

The higher a group’s anomaly score is, the more suspicious it is to be a fake reviewer group.

We evaluate our method on three real-world datasets from Yelp.com collected by [34]. YelpNYC contains reviews for restaurants in New York City. YelpCHI, collected by [35], comprises reviews for a set of hotels and restaurants in the Chicago area. YelpZip collects restaurant reviews from plenty of areas ordered by zip code. Details for each datasets is illustrated in Table I. Note that the labels in the datasets are near-ground-truth since they are generated by Yelp’s filtering algorithms.

We compare our methods with three baselines:

• •

  GraphStrainer by Ye et al. [36] first uses a graph-based method to discover target products and then detects fake reviewer groups via hierarchical clustering based on induced subnets. It does not use any handcrafted features.

• •

  ColluEagle by Wang et al.[37] is a markov random field based method exploiting group-level behavior features. It uses a density-based clustering approach[38] to extract candidate groups rather than deep approaches. Note that in our experiments we use vanilla ColluEagle without node prior.

• •

  DeFrauder by Dhawan et al.[17] first uses group indicators to extract groups, and then performs graph embedding via node2vec[39] to calculate density-based spam scores. DeFrauder exploits only group features, ignoring individual-level ones.

With only labels about the truthfulness of each review, it is hard to decide whether several people have worked collaboratively. We consider that the more fake reviewers in a group, the more suspicious the group is. Therefore, we define the precision of results as the ratio of fake reviewers in the group to all group members. In our experiments, we mark the user that has written at least one fake review as fake.

We consider detected groups with small sizes are much more likely to form by coincidence and also have only slight effects most of the time, while large fake reviewer groups are more common and harmful as mentioned before. Meanwhile, according to Dunbar’s number, a person’s social circle typically consists 15 good friends and 5 best friends. Considering the sum of these two sets of people as a small social clique, we set 20 as the lower bound for a relatively both large and tight fake reviewer group, and results with less than 20 members will not be taken into account. However, many existing group-level detecting methods[8, 9] can only uncover groups with less than 10 people, especially 2 to 5 people[9], and few detected groups have more than 20 members. Hence we compare the precision of the detected groups that ranked first among the outputs with more than 20 members to alleviate these issues in the baseline for better comparison. GroupStrainer is a cluster-based method, so we select the group that has the highest precision among those have more than 20 members.

For deep modularity networks in the clustering step, we analyze the effect of the number of clusters on YelpNYC and YelpZip dataset since it can significantly affect the group size in final results, which we consider crucial. We set $k$ in collapse regularization to 0.5, dropout to 0.5 and compare the precision of clustering results, skipping measuring indicators. The precision we calculate at this step is the mean precision value of the top 10 clusters, which is also the optimal situation for the next stage’s detection. Particularly, we present the cluster precision results on YelpNYC and YelpZip.

As demonstrated in Fig.4, when the cluster number is relatively small, the cluster precision goes up as the number of clusters gets greater. This could be due to that greater cluster numbers lead to smaller group sizes, which is beneficial for theoretical precision while its practical meanings are doubtful. When the cluster number is relatively large, the cluster precision goes down. The reason is that after clustering target product, we intersect the reviewer sets of the products to get candidate groups. Too many clusters will lead to fewer detected groups, and the more clusters we divide, the smaller the group size is. So a decrease in the total number could negatively affect the precision.

We illustrate the distribution of the size of detected groups in Fig.5, This shows that the sizes of most detected groups are among 20 to 150. After ranking the top 100 fake reviewer groups, the sizes gather around 20 to 100 as shown in Fig.6. This could be due to the difficulty in guaranteeing precision when extracting large sizes of groups.

Finally, we conduct extensive experiments on YelpNYC, YelpZip, YelpCHI with our baselines, and the precision result of the chosen group as presented in Table II. The results show that REAL achieves comprehensively good results on all three datasets.

Though ColluEagle achieves high precision in the results, it does not extract any group of large sizes and most groups it detects involve only 2 to 4 people. We cast doubt on the effectiveness of such results because in practice it is a rare case for a fake reviewer group to involve 2 to 4 people and groups with small sizes may be generated by chance. DeFrauder is able to detect many groups with large sizes, but its precision is relatively low compared with REAL. GroupStrainer performs comprehensively, but is still inferior to REAL. Meanwhile, we also notice that REAL’s precision is significantly lower in YelpZip compared with ColluEagle, we analyze that since YelpZip is especially larger than the other two, it could be that GCN does not use the same local filter to scan every node and the weights in the filters are the same for all neighboring nodes in the receptive field. As a result, REAL’s performance in large graphs is hindered.

Compared with baselines, REAL makes a satisfying balance between the group size and precision.