Towards Effective Detection of Ponzi schemes on Ethereum with Contract Runtime Behavior Graph

Listing 1 shows a code snippet of a typical Ponzi contract. The snippet comprises two functions, namely enter() and pay(), where enter() is responsible for receiving Ether from investors and pay() handles the redistribution of Ether. This contract promises investors very high return rates (Line 13) in exchange for their initial investment. The promised returns are paid out of new investments to attract additional investors until the scammers close up their scam and abscond with the illicit profits. Without legitimate earnings, a Ponzi scheme needs a steady stream of new investors to keep it running, otherwise, it will inevitably collapse and let the vast majority of participants bear the loss (Artzrouni, 2009).

Based on some previous studies (Bartoletti et al., 2020; SEC, 2019; Chen et al., 2021) and our analysis of known Ponzi contracts, we have developed explicit criteria for objectively identifying Ponzi contracts in our study. Our proposed criteria include:

• •

  A Ponzi contract must incorporate at least two explicit behavioral logics: investment and reward. This criterion excludes contracts that receive cryptocurrency but provide users with assets through external markets such as real-world trades or auctions that utilize cryptocurrency for payments.

• •

  The assets of a Ponzi contract must come from a multitude of investors rather than a specific source. This means that Ponzi contracts have no sources of income other than attracting investments. This criterion excludes contracts specifically designed to fulfill certain functions, such as enterprises distributing incentives to employees.

• •

  In a Ponzi contract, all the investors are promised rewards that are typically expected to exceed their initial investment, although the implementation of these rewards is contingent upon attracting further investments. In other words, as long as there are constant new investments, everyone can theoretically reap the rewards. This criterion excludes the contracts that are likely to be mistaken for Ponzi contracts, such as gambling and puzzle contracts. In such contracts, not all users are promised rewards as they would be in a Ponzi contract. (There are always losers in gambling or puzzle games.)

As shown in Table 1, we have selected some operations as taint sources to introduce taint data. These operations will push some external data into the stack or memory, such as CALLER, CALLVALUE, CALLDATALOAD, CALLDATACOPY, which are related to the transaction sender and arguments, and TIMESTAMP, BLOCKHASH, which are related to the blockchain environment. In addition, we also consider some operations related to the contract itself, such as BALANCE and ADDRESS. Data derived from these sources is marked as tainted, while other data is marked as untainted. Regarding taint sinks, we select some meaningful operations as the location to check the flow of taint data. These operations either take taint data as their arguments (e.g., GT, CALL and SSTORE), or load taint data and push it into the stack (e.g., MLOAD and SLOAD).

To achieve the taint propagation, we implement the taint engine that encompasses the components such as a taint stack, a taint memory, and a taint storage. Each slot of the taint stack contains a taint that marks the corresponding slot in the EVM stack. Since the EVM memory is a byte array, each taint in the taint memory is responsible for a byte in the EVM memory. Both the taint stack and taint memory are volatile regions that are freed and allocated at the start of each new transaction. In contrast, the EVM storage is non-volatile and stores state variables in key-value pairs. In these key-value pairs, a 32-byte address calculated from the state variable is stored as the key, and the state variable is stored as the value. We maintain the taint storage in the same structure, with the address of the state variable as the key and the taint as the value. As storage is non-volatile, the taint storage is kept until all transactions are completed, as part of the Ethereum world state. In general, when one operand of an arithmetic operation is tainted, the result of the operation is also tainted regardless of the other operands. The implementation of the taint engine enables us to capture and trace the data flow throughout the contract execution.

We gather the information obtained in the contract execution and construct a raw graph that integrates the control flow and data flow of the contract runtime. The nodes of the graph are the operations executed during runtime, and we add control flow and data flow edges as the graph edges. The control flow edges are categorized into six types, with the most common type being the adjacent edge. This edge connects two operations whose program counters differ by only 1, indicating that they are executed in a successive manner. The other types of control flow edges include the jump edge, which connects JUMP(I) and the operation executed after the jump, as well as the call, return, and creation edges that similarly connect the corresponding operation (e.g., CALL, RETURN, CREATE) and its successor. Regarding the data flow edges, we follow the principle of adding edges from taint sources to taint sinks, representing the propagation of taint data. There are eight kinds of data flow edges according to the taint sources in Table 1.

Figure 4(a) shows examples of the output graphs obtained from the dynamic taint analysis. The two graphs presented in Figure 4(a) are the result of invoking enter() and pay() within the contract shown in Listing 1. For the sake of clarity, only the nodes representing the taint sources and sinks that capture the main logic of the functions are included in these simplified graphs shown in Figure 4(a).

As outlined in Section 3.4, we generate transactions to invoke the contract and execute its designated behavior patterns. Each transaction triggers the contract to execute once, generating a corresponding graph structure. To maximize the activation of Ponzi scheme behavior patterns, we generate multiple transaction sequences for each contract. Consequently, numerous redundant graphs occur, corresponding to repeated contract executions, failed executions due to contract assertions or invalid inputs, and executions unrelated to Ponzi scheme behavior patterns.

To mitigate graph data redundancy, we adopted two strategies to prune the raw graphs. The first strategy involves pruning based on contract behavior. This entails removing graphs from failed executions and executions that are not related to Ponzi scheme behavior patterns. Specifically, if an execution does not contain the opcode CALLVALUE and CALLER, it is unlikely to represent an investment operation of a Ponzi scheme. Similarly, if the execution lacks comparison (such as LT, GT, EQ) on the state variables, it is unlikely to be a reward operation of a Ponzi scheme. Therefore, we remove from the graphs any executions that are neither investment nor reward behaviors. Additionally, we remove graphs without the SSTORE opcode, as it is one of the most commonly used opcodes to modify the contract state, and executions without SSTORE often indicate that they ended in failure. The second strategy involves pruning similar graphs. We compute the cosine similarity of node and edge features between each pair of graphs to assess their similarity. Graphs surpassing a specified threshold of similarity are deemed similar. Among similar graphs, we retain only one and discard the others.

After the invocation, a contract may correspond to multiple graphs, as each transaction can invoke the contract and generate a graph, as depicted in Figure 5. However, an individual graph may not be sufficient to fully capture the behavior of the contract. For instance, in Figure 4(a), each graph only depicts a single stage of the contract (i.e., the investment stage for enter() and the reward stage for pay()), and neither of these graphs alone can conclusively determine that it is a Ponzi contract. Moreover, the data flow of the contract is isolated among graphs. Since smart contracts have persistent variables, there may also be data flow across transactions, which cannot be captured by individual graphs.

To address these issues, we connect all graphs of the same contract sequentially using a new type of edge called connection edge. This creates a connected graph that can better represent the behavior of the contract across multiple transactions. Furthermore, we complete the across-transaction data flow among previously independent graphs using taint storage. The taint storage records the taint status of variables at the end of each transaction, and this information is used to propagate taints to subsequent transactions. With these enhancements, we are able to capture data flow that spans multiple transactions and more accurately analyze the behavior of smart contracts.

In the raw graph, nodes are distinguished by the type of operation they include. This results in each node being represented by a 139-dimensional one-hot vector (corresponding to 139 unique operations), with a single non-zero entry corresponding to the type of operation. However, one-hot vectors do not capture any information about the relationships between nodes in the graph, which are crucial for understanding its structure and properties. To improve the classification accuracy, we need better node embeddings that can capture these relationships.

We noticed that there is an introduction for each operation in the Ethereum Yellow Paper (Wood, 2022) as exemplified in Table 2. In Table 2, $\alpha$ represents the additional items placed on the stack, while $\delta$ represents the items removed from the stack (Wood, 2022). The description section explains how the operation works in text, and shows how it operates the data in the EVM in the formula. To embed the nodes, we first remove the formula in the description section and keep only the text explanation to preserve the functional information of the operation. As shown in Figure 5, for each node, we use Doc2Vec (Le and Mikolov, 2014), a model for generating embeddings of variable-length pieces of text, to convert the text explanation into a 100-dimensional vector, which we stitch together with $\alpha$ and $\delta$ to form the node feature. After that, we have completed the construction of CRBG which will be labeled for model training later. Figure 4(b) shows the constructed CRBG after the raw graphs in Figure 4(a) were preprocessed. The CRBG in Figure 4(b) has better node embeddings, more comprehensive contract runtime information, and can better characterize the contracts.

We use CRBG ($\mathcal{G}$) as the input graph which contains nodes $\mathcal{V}=\{1,...,n\}$ and edges $\mathcal{E}$. The node features matrix X has a dimension of $(|\mathcal{V}|,102)$, where each node is represented by a 102-dimensional feature vector. The edge index I has a dimension of $(2,|\mathcal{E}|)$, where each column corresponds to an edge and contains the indices of the nodes that the edge connects. The edge features matrix E has a dimension of $(|\mathcal{E}|,15)$, where each edge is represented by a 15-dimensional feature vector. $|\mathcal{V}|$ and $|\mathcal{E}|$ represent the number of nodes and edges in $\mathcal{G}$.

In graph embedding learning, we choose Graph Attention Networks (GAT) as the component of GNN convolutional layers. GAT performs the aggregation based on the self-attention mechanism, i.e., calculating the weights between nodes and edges through learnable weight matrices W and $\textbf{W}_{e}$, so that each node can be weighted and aggregated according to the characteristics of its surrounding nodes. Since CRBG has multi-dimensional edge features, the attention coefficients $\alpha_{i,j}$ in the self-attention mechanism are computed as:

By calculating the weight between nodes, the weighted sum of the adjacent nodes of node $i$ can be obtained:

We set two GAT layers and use ReLU in the middle for nonlinearly transforming the node features in order to better handle the nonlinear relationship of data and increase the expressiveness of the network. We utilize mean-pooling to aggregate the node features and obtain the global feature representation of the graph.

The classifier comprises a dropout and a fully connected layer (FC). The dropout randomly sets a fraction of the output of neurons to zero, which helps prevent overfitting and improves the model’s generalization ability. The purpose of a fully connected layer is to learn non-linear combinations of the features in the input data, allowing the model to make more accurate predictions. We input the global feature representation into the classifier and obtain the predicted class label for the graph.

We utilize the dataset in XBlock (zhijie, 2023) provided by Zheng et al. (Zheng et al., 2023), obtained through crawling Etherscan (Etherscan, 2023) and manual cross-checking. This ground-truth dataset comprises 6,498 smart contracts, with 314 identified as smart Ponzi schemes. These smart contracts range from height 0 to height 7,500,000, and undergo a manual cross-check procedure to ensure the accuracy of their labels.

We use the following evaluation metrics to measure the effectiveness of our approach.

Precision measures the proportion of true positive predictions made by the approach out of all positive predictions: Precision = TP / (TP + FP). Recall measures the proportion of true positive predictions made by the approach out of all actual positive instances in the dataset: Recall = TP / (TP + FN). F1-score is the harmonic mean of Precision and Recall, providing a single measure of the approach’s overall performance: F1-score = 2 $\times$ Precision $\times$ Recall / (Precision + Recall)

We evaluated the effectiveness of PonziGuard and compared it with the studies of Chen et al. (Chen et al., 2018), Yu et al. (Yu et al., 2021), SADPonzi (Chen et al., 2021), MulCas (Zheng et al., 2023) and SourceP (Lu et al., 2024). Chen et al. (Chen et al., 2018) detect Ponzi contracts using XGBoost mainly based on the opcode frequency, and in this paper we refer to their work as OpML. Yu et al. (Yu et al., 2021) utilize the transactions on Ethereum to identify Ponzi contracts, and in this paper we refer to their work as TxML. SADPonzi detects Ponzi contracts based on symbolic execution. MulCas extracts contract features from multiple views and detects Ponzi schemes through multi-view training and ensemble. SourceP trains the classification model by converting contract source code into Abstract Syntax Trees (AST) to extract data flow information. (Chen et al., 2018; Zheng et al., 2023; Lu et al., 2024) represent state-of-the-art (SOTA) operation/source code-based machine learning approaches. (Yu et al., 2021) represents SOTA transaction-based machine learning approaches, and (Chen et al., 2021) represents SOTA rule-based approaches.

In our approach, we generated 6,498 graphs from 6,498 contracts in the dataset for model training and testing. While we used the same dataset in the comparative experiment, different approaches processed the data differently. For example, in the approach of Chen et al. (Chen et al., 2018), we compiled the 6,498 contracts into bytecode and counted the opcode frequency as inputs for model training and testing. In the approach of Yu et al. (Yu et al., 2021), we collected the transactions of these contracts on Ethereum and performed a random selection process to obtain a transaction network as input. The contract bytecode could be directly applied by the symbolic execution tool of SADPonzi. For the machine learning-based approaches, we randomly divided the dataset into 5 folds and performed K-fold cross-validation. The mean values of the evaluation metrics across the K models, as well as their corresponding standard deviations, were calculated to measure the average performances. As shown in Table 3, PonziGuard outperformed all the baselines on the test set, achieving 96.9% precision, 98.2% recall, and 97.5% F1-score. We believe that the poor performance of the state-of-the-art approaches can be attributed to the fact that static information cannot characterize the Ponzi contracts (OpML, TxML, MulCas and SourceP), and not all Ponzi contracts conform to the pre-defined behavior patterns (SADPonzi).

To evaluate the impact of graph pruning and graph joining, on the performance of CRBG, we designed two ablation settings. In each setting, we omitted one of these steps respectively. We then compared the size of CRBG generated in these two ablation settings and the corresponding experimental results on the same dataset, which is shown in Figure 10 and Figure 10.

For the CRBG without the graph pruning operation, as shown in Figure 10, the average number of nodes and edges per graph increased by nearly 30%. This added significant redundant information, leading to lower precision, recall, and F1-score compared to the original CRBG. Additionally, the larger graph sizes increased the model’s training time. For the CRBG without the subgraph joining operation, as shown in Figure 10, the average number of nodes and edges per graph was significantly reduced to nearly one-fifth of the original CRBG. However, as discussed in Section 3.5, these smaller subgraphs failed to fully capture the contract’s behavior, resulting in a decrease in performance metrics: precision, recall, and F1-score dropped by 5.9%, 10.2% and 8.0%, respectively.

To gain a better understanding of the effectiveness of control and data flow in CRBG, we performed an ablation study by configuring PonziGuard in four distinct modes: data flow only (DF), control flow only (CF), both (DF+CF), and neither (NE). In DF mode, we removed control flow edges and only kept data flow edges in CRBG. On the contrary, in CF mode, we removed data flow edges and only kept control flow edges. In NE mode, we removed both data flow edges and control flow edges in CRBG. The mode DF+CF is the native PonziGuard which includes both control and data flow.

Figure 11 shows the performance of these four modes after 5-fold cross-validation on the same dataset. Compared to the native PonziGuard baseline, there were drops of 9.3%, 15%, and 12.2% in the evaluation metrics of the CF mode. In the DF mode, the evaluation metrics decreased to a greater extent compared to the native PonziGuard baseline, with a drop of 13.5%, 24.2%, and 19.3%, respectively. Undoubtedly, NE mode exhibited the worst performance, with a significant drop of 26.4%, 42.1%, and 35.3%, respectively. The main reason for the poor performance of the CF mode is that, with control flow only, PonziGuard cannot capture the flow of investors’ investments in contracts. Therefore, some contracts with Ether redistribution logic may be misreported. On the other hand, the lack of control flow in the DF mode results in the loss of contract context information, such as the functions and order in which variables are used. This ablation study highlights that both control flow and data flow are crucial in capturing the behavioral patterns of Ponzi contracts, and the gathering of this runtime information significantly improves the performance of PonziGuard.

In Section 3.5, we utilized Doc2Vec to enhance node embeddings in CRBG based on the operation descriptions from the Ethereum Yellow Paper. We believe that the semantic information conveyed in these descriptions is representative and can capture the relationships between the nodes in CRBG. We conducted a comparative experiment to evaluate the efficacy of the node embeddings we enhanced. In this comparative experiment, one model was trained on the dataset described in Section 5.2.1 using our enhanced node embeddings (Model with enhanced node embeddings, abbreviated as MEE). In contrast, another model was trained on the same dataset, but replacing our node embeddings with one-hot vectors (Model with one-hot vectors as node embeddings, abbreviated as MOE). We used 80% of the dataset for training and 20% for testing. As shown in Table 4, compared with the one-hot vectors as node embeddings (Test 1), the node embeddings based on the operation description (Test 2) performed better in the evaluation metrics. It demonstrates that the node embeddings generated from operation descriptions capture the underlying semantics, leading to a better understanding of the graph’s structure and properties, which accounts for this performance improvement.

To ascertain that the performance improvement is primarily attributed to the semantics themselves, rather than the way the semantics are described, we conducted additional analysis. We rewrote the operation descriptions in the Ethereum Yellow Paper with two principles while retaining the original semantics. The first principle is using synonyms substitution. By substituting words with their synonyms, we retained the original semantics while using a different expression. Another principle is changing sentence structure or grammar. This can be done by using different sentence patterns, altering the word order, or adjusting the placement of clauses. For instance, as shown in Table 5, the description for JUMPI in Table 2 can be rewritten as ”Conditionally change the instruction pointer” and ”Alter the program counter based on the condition” according to these two principles. Based on these two alternative description rewriting principles, we re-extracted the node embeddings and created two variants of the test set (Variant 1 and Variant 2). Then, we evaluated our model (MEE, the model trained with enhanced node embeddings) on these two variant test sets. As shown in Table 4, our model exhibited similar performance on the variant test sets (Test 3 and 4) compared to the native test set (Test 2), suggesting that the improvement in model performance is primarily attributed to the semantic information rather than the specific way in which the semantics are conveyed.

Firstly, we ran the Geth client with the option: sync-mode-full to synchronize with the Ethereum Mainnet. The number of smart contracts has experienced explosive growth in recent years (about one million per quarter (Alchemy, 2022)), which is a significant amount for our approach based on runtime information. Therefore, we set the synchronization time until January 2022 (approximately 14,000,000 blocks), only as a preliminary experiment to verify the performance of PonziGuard in real-world scenarios. Then, we replaced the native EVM with our instrumented EVM and integrated the dynamic taint engine. We re-executed every transaction on the synchronized blockchain from the genesis block, which is a time-consuming process. Finally, we fed the generated graphs into our GNN model for prediction. As a result, PonziGuard successfully identified 805 Ponzi contracts on Ethereum Mainnet, out of which 497 contracts have accessible source code on Etherscan (Etherscan, 2023). We randomly selected 50 contracts³³3https://github.com/PonziDetection/PonziGuard/tree/main/dataset/Result/verified from these 497 contracts and conducted a manual examination through Remix (Project, 2023), a solidity IDE, to ensure they meet our predefined criteria for Ponzi contracts, resulting in a 100% true positive rate. To gain deeper insights into these 805 Ponzi contracts, we conducted further analysis using the data collected from Etherscan in the remainder of this section.

Creation Time of Ponzi Contracts. Figure 13 shows the distribution of these 805 Ponzi contracts. Ponzi schemes started appearing on Ethereum as early as 2015. Subsequently, the rapid development of Ethereum led to a significant growth of Ponzi contracts during the years 2016-2019. Then, we witnessed a brief recession in Ponzi schemes possibly linked to the impact of the COVID-19 pandemic (Lehman, 2021). The global crypto mining boom in 2021 (Drakopoulos, 2021) resulted in another minor peak in Ponzi schemes. With the increasing popularity of various tokens on Ethereum, ERC-20 Tokens for instance, we anticipate another peak in Ponzi schemes on Ethereum in the near future.

Lifetime of Ponzi Contracts We regard the time from the creation of a Ponzi contract to its last transaction as its lifetime. We investigated the lifetime of these 805 Ponzi contracts, as shown in Figure 13. While some of these contracts remain active in 2023⁴⁴4For instance: 0xa90be2201bfed97587a2a17949e8624eafe51d13 and 0xf8f04b23dace12841343ecf0e06124354515cc42, the majority of Ponzi contracts have a lifetime of less than three months, and their average lifetime is about seven months. As for the short lifetime of the Ponzi, some ended because the scam failed to attract new investors, resulting in its collapse. Others were ended because the scam owner intentionally triggered the self-destruct function of the contracts and absconded with the funds. These findings indicate that Ponzi contracts are likely to collapse within a short period of time, and most of their users will not be unable to reclaim their promised returns.

Financial Impact We analyzed the financial impact of the 805 Ponzi contracts identified by PonziGuard on Ethereum Mainnet by aggregating their transactions and the inflow of Ether. Figure 15 shows their monthly distribution, revealing a positive correlation between the inflow of Ether and the number of transactions of the contracts. The peak was in February 2018, when a total of 117,953 Ether flowed into Ponzi contracts, equivalent to $108 million at the exchange rate of that time. From January 2015 to July 2023, 615,483 transactions, totaling 281,700 Ether, flowed into Ponzi contracts. At the current exchange rate, the value of these tokens can reach as high as $500 million. It is also evident that, in recent years, the involvement of Ether may not be substantial in a Ponzi scheme, as some of them began adopting ERC tokens for investments and rewards. However, it is important to note that such kinds of Ponzi contracts still meet the criteria outlined in Section 2.2, and our method remains effective in identifying them⁵⁵5Evidenced by the example of 0xb3836d31d43d315ba74c21aad3818f9378256152.

To ascertain whether new Ponzi schemes still keep emerging and whether our tool can detect them effectively, we collected the source code of about 10,000 smart contracts recently deployed in March 2024 from Etherescan (Etherscan, 2023) and employed PonziGuard for detection. After manual confirmation, we identified 15 Ponzi schemes among these 10,000 recently deployed contracts, and their address are available at our GitHub repository⁶⁶6https://github.com/PonziDetection/PonziGuard/tree/main/dataset/Result. As shown in Figure 15, most of these Ponzi contracts were newly deployed and have only one transaction (creation transaction) on them, showcasing the ability of PonziGuard to uncover 0-day Ponzi schemes.

For the experiment described in Section 5.2, the contracts were executed in an independent instrumented EVM with the taint engine. To evaluate the time overhead, we generated 1,000 transactions for a contract and sent them along with the contract to both the native EVM and the instrumented EVM separately. To accurately assess the time overhead, we repeated the process 10 times and recorded the average time it took to process these transactions. The results, shown in Figure 17, indicate that when the processing of 1,000 transactions was completed, the average overhead of the instrumented EVM reached a maximum of approximately 30.2%.

In the experiment described in Section 5.5, we conducted re-execution of historical transactions on the synchronized blockchain. To evaluate the time overhead, we re-executed the transactions of the first 500,000 blocks on the synchronized blockchain using both the native Geth and the Geth modified by PonziGuard separately. To accurately assess the time overhead, we repeated the process 10 times and recorded the average time it took to complete the re-execution. As shown in Figure 17, when it comes to 500,000 blocks, the time overhead amounts to 25.5%, which is smaller than the overhead on the ground-truth dataset. This can be attributed to the fact that re-execution involves additional reading and verification operations on the blockchain, in addition to the time consumed by contract execution.