Blockchain Systems, Technologies and Applications: A Methodology Perspective

The infrastructure layer and blockchain layer are interrelated and interact on each other cooperatively, while the detailed procedures can vary among different blockchain systems. However, for all blockchain systems, they have to follow the following basic steps, as shown in Fig. 1. Firstly, blockchain clients generate transactions and broadcast them to the P2P network in Step I. More than one nodes may bundle different subset of unverified transactions into their candidate blocks in Step II. Afterwards, all nodes perform incentive and consensus mechanisms to find the winner whose candidate block would be announced as a new block in Step III. Then, the new block will be inserted into all node’s local ledgers in Step IV. This mechanism connects multiple blocks together and builds a chronological chain. In particular, the process of hashing a new block always contains metadata about the hash value of the previous block, which makes the linked data highly non-modifiable. At last, after the transaction is stored in the blockchain, the client can request the consensus network to confirm whether a transaction is in the blockchain.

Through this workflow, the transaction is finally agreed by the majority of nodes and recorded in blockchain, where malicious nodes cannot subvert the consensus results. This decentralized architecture ensures robust and safe operations on the blockchain, with the advantages of tamper resistance and no single point of failure. It can be seen from the Fig. 1 that each blockchain node in the blockchain system undertakes all or part of functions, such as communication between nodes, maintenance of P2P network, and computation of consensus mechanism. Thus, the underlying communication, network and computation is crucial to establish effective and secure blockchain system. This encourages us to study how communication, networking and computing affect blockchain systems. Fortunately, some classic methodologies can provide good ideas, such as stochastic process for block generation and nodes communication, machine learning for P2P network performance improving, and optimization theory for resource allocation. Therefore, it is feasible and valuable to explore the interaction process of communication, network and computing and their impact on the blockchain system from the perspective of methodology, and even can provides help for revealing the essential problems in the operation process of the blockchain system.

Recognising the wide applications of blockchain technology, a novel survey paper can help researchers in various fields to build good foundations on the subject to guide actual developments. Recently, several work have reviewed the advanced development of blockchain from various views.

For security and privacy, T. Salman et al. in [13] present blockchain-based security services in authentication, confidentiality, privacy and access control, etc. N. Waheed et al. in [17] summarize the research efforts of using machine learning algorithm and blockchain technology to address security and privacy problems in the field of Internet of Things in the past few years. M. Conti et al. in [18] focus on the security and privacy threats of Bitcoin, and discusses the feasibility and limitations of potential solutions. M. Saad et al. in [19] focus on how attacks effect public blockchain and discusses the relationships between a sequence of possible attacks.

Besides, as the core of blockchain, consensus determines the performance and security of the blockchain in many ways, M. S. Ferdous et al. in [20] utilize comprehensive taxonomy of properties to analyze a wide range of consensus algorithms, and examines in detail the meaning of the different problems that are still prevalent in the consensus algorithm. W. Wang et al. in [21] review the state of the art consensus protocols and game theory in mining strategy management.

In the context of artificial intelligence (AI), Y. Liu et al. in [22] discuss feasible solutions integrating blockchain and machine learning for communications and networking system. Y. Xiao et al. in [23] introduce the classic theory of fault tolerance and analyzes blockchain consensus protocols using a five-component framework.

For the scalability of the blockchain, J. Xie et al. in [24] study the scalability of the blockchain system, analyzes the scalability from the perspective of throughput, storage and network, and introduces the existing enabling technology of the scalable blockchain system. H. T. M. Gamage et al. in [25] discuss issues of the existing blockchains such as 51% attack, nothing-at-stake problem together with improvements for the scalability issues in current blockchains. R. Belchior et al. in [26] study cryptocurrency-directed interoperability approaches, blockchain Engines and blockchain connectors, providing a holistic overview of blockchain interoperability.

The integration of blockchain and 5G has become a mainstream trend, D. C. Nguyen et al. in [27] provide the latest survey on the integration of blockchain with 5G networks and other networks. It gives an extensive discussion about the potential of blockchain for enabling key technologies of 5G, and further explores and analyzes the opportunities that blockchain may give important 5G services. G. Yu et al. in [28] study the sharding problem in the blockchain, mainly including providing detailed comparison and quantitative evaluation of the main sharding mechanisms, as well as analysis of the characteristics and limitations of existing solutions. Moreover, by enabling the integration of blockchain and other advanced technologies, some of work explore potential applications and research challenges in IoT [29, 30], smart city [31], cloud computing [32], edge computing [33], and fog computing[34].

From the perspective of mathematical tools, Z. Liu et al. in [35] provide reviews and analyses using game theory in detail to deal with a variety of problems regarding security, mining management and blockchain applications. However, some classic mathematical methods used in blockchain are not limited to game theory, but also include machine learning, cryptography, and so on. Moreover, a survey which provides a comprehensive overview on theoretical model, network service and management, and application for blockchain system on a methodology perspective is even more needed.

The existing surveys mainly focus on blockchain architecture, blockchain protocol algorithm, the integration of blockchain with other network technologies, etc. Besides, to the best of our knowledge, there is very few survey provides comprehensive reviews to elaborate the impact of communication networks and computing from methodology perspective. However, as a system science, blockchain requires researchers with multidisciplinary knowledge background in communication, network, computing and cryptography disciplines to understand the operating environment and principles of blockchain from different dimensions, so as to make rational use of blockchain technology to improve innovative service applications in 5G/B5G, IoT and other fields.

This motivates us to review the state of the art of blockchain, to reveal the reason why a blockchain system needs communication, network and computing and how them interact to each other. Furthermore, to facilitate the deployment of blockchain-based applications, how to carry out mathematical methods to define a specific blockchain problem is a concern that many researchers deserve to pay attention to, but there is no survey specifically discussing the use of blockchain from the perspective of methodology such as stochastic process, cryptography and so on. Motivated by this limitation, different from the previous surveys that focus on blockchain application such as IoT, smart city and edge computing or blockchain technology such as security and privacy, consensus algorithm and resource management, we overview the theoretical model for blockchain fundamentals understanding, and the design of network services for blockchain-based mechanisms and algorithms from a methodology perspective involving of stochastic process, game theory, optimization, machine learning, and cryptography, as well as summarizing the advantages and limitations of these methods. Moreover, this paper reviews some case studies and challenges applying blockchain in IoT, providing feasible guidance and potential research problems.

The organization of this article is as follows. Section II discusses the research activity, challenges and roadmap of blockchain system. Section III-VII analysis the application of stochastic processes, game theory, optimization theory, machine learning, and cryptography for mining pool management, security strategy, resource management and so on in blockchain, respectively. Section VIII discusses main issues of blockchain and its improvements. Finally, Section IX concludes this paper.

This article aims to provide a comprehensive overview on theoretical model, network service and management, and application for blockchain system on a methodology perspective. In order to illustrate the relationship in methodology, use cases and topic, a technical roadmap is shown in Fig. 3. First, we introduce the calssic methodology to investigate running process of blockchain to formulate and analyze. Then illustrate the use cases for blockchain theoretical model, blockchain technology design enabling network service, and how to use blockchain as the fundamental of industry vertical applications effectively. Accordingly, we review the state of the art of blockchain system for performance and security analysis, resource management, incentive mechanism, ect, and some remaining problems are summarized and discussed.

In practical blockchain system, there exist many random behaviors such as block generation time, confirmation delay, chain growth rate, and forking. As mentioned before, it is a nature design to adopt stochastic process to model these random behaviors systematically, which can help us to formulate the blockchain operation process effectively and accurately. Next, we discuss how to use stochastic process in blockchain system on the typical issues of consensus, security and deployment. In order to formulate the consensus process in blockchain system as a stochastic process, it is necessary to define a metric (like the cumulative block in PoW, or the cumulative weight in DAG) to indicate the consensus state at the different time. Moreover, if it satisfies Markov properties, the consensus process can be formulated as a Markov chain model with one-step transition probability $P=P\{X_{n+1}=i_{n+1}|X_{n+1}=i_{n}\}$.

As shown in Fig. 4, we use Markov chain to model the consensus process. In this process, $S_{i}(i=1,2...,m)$ represents the state in the consensus process, and $P_{i,j}$ is the one-step transition probability in the Markov chain mode. Accordingly, we can learn the consensus process and gradually understand the inner-action. Furthermore, it is also useful to analyze the malicious forking attack for security.

As the most famous consensus process, the PoW-based mining task proposed by bitcoin is a competition among minors, where the winner has the right to generate a new block to obtain an amount of reward. For malicious purposes like double-spending in the blockchain, forking attack launched by the malicious node which generates blocks to build a parasite chain privately, it would succeed if the parasite chain is longer than the main chain built by the honest node due to Longest-Chain-Rule (LCR). Indeed, this attack can be treated as a competition between honest and malicious nodes. Particularly, without any malicious node, this competition is also the consensus process formulated previously. Therefore, the competition for block generation can be modeled as a Poisson process to study the relationship between honest and malicious nodes, which is shown in Fig. 5. The malicious nodes compete against the honest nodes to generate the block. The node which has the biggest probability will have the right to generate the block. Accordingly, the factors affecting the vulnerability of blockchain can be known, and the successful probability of malicious node can be determined. As a result, the theoretical insight can be provided to resist forking attack, optimize consensus mechanisms, and improve network security.

Like the classic base station deployment problem in heterogeneous networks, deployment of blockchain funtion node (or called as full node in some literatures) can be solved using a stochastic method/stochastic methods in the same manner. The blockchain system is decentralized which is composed of multiple distributed blockchain nodes, their geographical distribution can be modeled as a Homogeneous Poisson Point Process (HPPP). In this way, we can construct an effective blockchain system architecture, and analyze the impact of blockchain node distribution on the communication throughput, SNR, security and other network metrics. Therefore, a valuable theoretical guidance is developed to optimize the blockchain node deployment in order to imporve the system performance.

In this section, we will use our previous work [62] as an example to introduce how to model the classic blockchain problem as a stochastic process. As shown in Fig. 6, Tangle is a DAG based distributed ledger for recording transactions.

DAG consensus is a typical voting mechanism to accumulate weight for consensus, which allows the new transaction to randomly select two unselected transactions that are called as tips. Therefore, according to the random selection in DAG consensus, selected times of the observed transaction within the time period $[0,t]$ is a random variable of $t$ based on the assumption that the arrival time of the observed transaction is 0. Thus, the cumulative weight of the observed transaction is its own weight plus the overall number of transactions which select it (the weight of each transaction is assumed to be 1). Therefore, the cumulative weight $W(t)$ is a random variable with time $t$ ($\{W(t),t\in[0,\infty]\}$), which is a stochastic process. Meanwhile, $L(t)$ is the number of tips at time $t$, and it is also a stochastic process because it is affected by the random selection as well as $W(t)$.

In summary, the future states of $L(t+1)$ and $W(t+1)$ are determined by their current states ($L(t)$ and $W(t)$) only, which satisfies Markov properties. Therefore, this consensus process can be formulated as a Markov chain. Due to the consideration that the new transaction arrives slowly resulting in a low network load, the system state is defined as $\{W(t),L(t)\}$ that is modeled as a discrete Markov chain $\{W(k),L(k)\},k=0,1,2,...,\infty$.

When a new transaction $x$ arrives, the change in system state can be expressed as

where the $a_{x}=1$ stands for the situation that the observed transaction has been approved by an incoming new transaction, and $a_{x}=0$ stands for the situation that the observed transaction has not been approved. Since the new transaction should select two unselected transactions randomly, thus (3.2) indicates that the new transaction replaces two unselected transactions as a new one.

Therefore, the corresponding one-step transition probabilities and Markov chain can be shown as Fig. 7.

For example, if the network is from high to low with Unsteady State, the one-step transition probabilities can be expressed as:

This model is for low network load case, and we can also obtain the modelling for high network load case in the same manner. According to this Markov chain model, it is possible to provide a theoretical guidance for the blockchain implementation, which can help us to analyze the key performance indicators in terms of cumulative weight and confirmation delay under different network loads, and it is able to evaluate the security performance based on the understanding the impact of network loads.

Due to the randomness of the blockchain system, stochastic methods is commonly used for formulation, which can accurately describe the distribution of blockchain nodes, the arrival of transactions, the behavior of blockchain users and etc. Through these mathematical modeling, researchers can track the dynamic evolution of the blockchain system, and further analyze of the consensus process, throughput, and security performance. Accordingly, the corresponding theoretical basis can be provided for performance improving, such as malicious attack preventing, and blockchain application accelerating.

Although stochastic process has been widely used in the literature, there are still issues that should be considered and improved in the future.

1. 1.

   Most existing researches formulate the blockchain system as a certain stochastic process assuming a simple model such as the Poisson distribution. However, in the actual environment, the blockchain running process usually is more complicated, and how to abstract the common random variables accurately without lossing generality should be well studied. In another word, the mathematical formulation must accurately describe the blockchain system in practice, while considering the property, complexity and constraint in the view of theoretical approach.

2. 2.

   Currently, some typical issues in a few specific scenarios have been widely investigated. In contrast, a generalized stochastic model is in need to describe the whole blockchain system. In the future, we should further consider how to use stochastic process to model an end-to-end blockchain system model, which can systematical study the actions of blockchain user and the system performance. In addition, understanding the interactions between various functions of hash operation, cache, consensus, communication network and smart contract is another direction for future research.

Many blockchain problems can be structured as a game based on game theory. Through the analysis of the optimal strategy and equilibrium solution, we can investigate the operation process of blockchain mechanisms and the behavior of blockchain users to optimize the systematic performance accordingly.

Considering the malicious characteristic, a miner launches an attack to increase his/her own winning probability while unfairly reducing the wining probability of others. In fact, any selfish and rational miner would like to maximize its own overall reward, which is determined by the environment feedbacks, cost and successful attacking probability simultaneously. Therefore, a miner must consider all possible re-actions of others to choose the strategy that is most beneficial to itself in this typical non-cooperative game. Therefore, the miner can be treated as the player, its strategy is whether to launch an attacking, and the reward function is the expected reward if the attacking succeeds minus the cost for attacking. In this manner, we can formulate this mining process as a game shown in Fig. 8, to study the impact of miners’ strategies (to be honest or malicious) on the blockchain network. Based on the analysis and equilibrium solution, a game model could employed and to provide a theoretical guidance to optimize the consensus process in order to improve the security (refrain from launching the attacking action).

During the mining process, a miner should allocate certain amount of computing resources to increase the wining probability to get the right that generates a new block with corresponding reward. Meanwhile, the more computing resources consumed, the higher cost would be generated. It means there is an effective trade-off between cost and benefit needs to be made. Accordingly, this problem can be also formulated as a game to study the interaction among miners and analyze the best strategy. for instance, game theory is often used to study equilibrium-based strategy, which guarantees the optimal reward of each miner while avoiding the meaningless mining competition caused by greedy resource allocation. Fig. 9 is a game theory model based on auction, the miners will bid for the computing resource, and the computing service providers will provide different computing resource. The more bid, the more resource. Therefore, the miner need to balance the cost and revenue.

Using mobile edge computing in blockchain networks, the miner can offload its mining task to edge server, which can solve the limitation of computing resource of blockchain users to extend the blockchain application in wireless scenarios effectively. To encourage the miner to offload reasonably and motivate edge server to process effectively, the miner should pay an amount of payment to edge server for offloading. As shown in Fig. 10, the miners will offload some mining task to mobile edge. The more computing resource the miner buy, the greater the probability of successfully minings. Stackelberg game can be employed based on the concerning of resource allocation as mentioned above. If addressing the mining task assignment (the association between edge server and miner), auction model is a common approach.

In this section, we will use our previous work [73] as an example to introduce how to motivate honest actions of participants in blockchain-based scenarios.

In the typical MEC enabled WBN, controlled by the MEC manager, edge servers are just treated as network resources providers, including computational resources and storage resources. However, the MEC server manager may become a central node that is independent of the blockchain system, thereby undermining the distributed nature of the blockchain system and further damaging its security. To this end, in this example, the underlying P2P network consists of edge servers, who are treated as blockchain miners, undertaking blockchain functionality operations and earning transaction fees, while IoT devices are treated as blockchain users, who only need to upload transactions to blockchain with specified transaction rate requirements. As shown in Fig. 11, the blockchain users submit transactions to blockchain miners, and then the blockchain miners execute the mining task and successfully generate a block. Finally, it will be added to the local ledger and broadcast to peers and this operations will also be performed by other blockchain miners once they have verified it as a valid block.

At blockchain users’ side, the utility of a blockchain user $f_{j}$ includes the satisfaction degree and incentive cost, i.e., the transaction fee. Thus, in order to maximize the utility by requiring considerable transaction rate $\gamma_{j}$, the optimization problem for $f_{j}$ can be expressed as follows:

where $S_{f_{j}}(\gamma_{j})$ is the satisfaction degree, $C_{f_{j}}(\gamma_{j})$ is the transaction fees and $\Gamma_{max}$ is the maximum transaction rate blockchain system can afford.

At the blockchain miners’ side, the utility of them is defined as charged transaction fees minus computational resources consumption. Thus, to maximize their revenue, the optimization problem can be expressed as

where $N$ is the number of blockchain users, $S_{l}(\sum_{j=1}^{N}\gamma_{j},\beta)$ is the earning by publishing $\sum_{j=1}^{N}\gamma_{j}$ transactions per hour and $C_{l}(\sum_{j=1}^{N}\gamma_{j})$ is the corresponding cost of resources consumption.

With blockchain miners acting as the leader while blockchain users acting as followers, a single-leader-multiple-followers Stackelberg game can be used to model interaction between them. Based on the Karush-Kuhn-Tucker (KKT) conditions and backward induction method, a distributed algorithm can be designed to reach the optimal strategy $(\beta^{*},\gamma_{j}^{*})$ in an iterative manner.

As an analysis tool, game theory is widely used to study security, mining, and resource allocation problems in blockchain networks. A game model can be built by capturing the characteristic of the addressed problem in terms of the role of blockchain users, behavior of blockchain decision-maker and the performance of blockchain system. Thus, the game modling can help researchers understand the impact of different strategy and analyze the optimal strategy based on equilibrium solution.

Meanwhile, some problems are still remaining to be addressed as follows.

1. 1.

   Some necessary information should be collected and exchanged in game theory, for example, the selling price of edge server for mining task offloading and the computing resources used by miners for mining. Schedule the communication resource into the game theory need to be well investigated.

2. 2.

   Furthermore, to achieve the equilibrium solution, multi-round iterations in game theory are needed usually, but the corresponding overhead should be considered. The overhead which cause commnuication delay, would generate significant impact on the performance and the security of blockchain system. Next, efficient and light-weight game theoretic mechanism for blockchain system should be investigated.

3. 3.

   Rationality and selfishness are the basic assumptions in game theory. However, these assumptions might be invalid especially for the malicious attacker, whose purpose is to launch an attacking to ruin the blockchain system regardless of the cost. Therefore, how to understand and study this extreme malicious action in order to improve security and privacy should be studied in the future.

Optimization theory is widely used in various fields, which help us to adjust the parameters, schedule the resource and determine the decision improving the system performance in a distributed/centralized manner. In blockchain, it can be used to manage the mining, improve security, and jointly optimize the blockchain with other technologies. The mining management optimization is a typical problem in blockchain. As well known, the mining reward for miners is an important factor to encourage the contribution of computing resource. Therefore, maximizing the reward and promoting the accomplish of the consensus process is a popular topic in mining management. In order to increase the successful mining probability, miners usually choose to join the mining pool. Generally, as long as any miner in the pool succeeds in mining, the mining reward will be distributed to each miner in the pool. Different mining pools may adopt different reward mechanisms. Therefore, the miner should consider how to choose the optimal pool selection strategy to optimize its reward, and this problem can be transformed into a mathematical optimization problem. For this optimization problem, the mining reward obtained by miners under different reward mechanisms can be formulated as the objective function, the variable is the miner’s pool selection strategy, and the constraints are determined by the actual situation. Through this mining pool selection optimization problem, we can study the impact of different reward mechanisms on the blockchain network in term of the objective function, and determine the optimal selection under the constraints.

When miners join the mining pool and cooperate with others, the competition among miners becomes the competition among the mining pools. In order to win for mining reward, some mining pools may choose to take the Block Withholding (BWH) attacking on other mining pools. As shown in Fig. 12, the pool A will allocate a part of computing power $\alpha$ to another pool B for attacking. The greater computing power consumed by the attacker, the greater the malicious impact on other pools would be happened. The malicious impacts would decline the computing power for consensus and is costly to the attacker itself. As a result, the attacker should choose the optimal computing power for attacking, which can be considered as an optimization problem. In this case, the objective function is to maximize the mining reward as well as successful attacking probability with the computing power for attacking as the variable. Through the optimization analysis, we can know the optimal attacking strategy, which is the baseline to analyze and design the consensus process for security based on the understanding the malicious action of the attacker.

As discussed before, mining task offloading from the miners to the edge servers can be also formulated as an optimization problem with resource allocation. In this case, the miner should choose an appropriate offloading strategy to determine whether to offload, or to which edge server to offload, and how many resources (computing, communication and cache) the edge server allocates to the offloading task. Using this optimal offloading formulation, we can analyze the impact of the offloading on the performance and security of the blockchain network, and the optimal solution is to provide a theoretical guidance for the blockchain development and application.

In this section, we use the previous work [86] as an example to introduce how to perform optimization in blockchain. In this paper, to improve the security and privacy of D2D (device to device) communication, the authors introduce a new distributed and secure data sharing framework called D2D blockchain. The simplified procedure for transaction relaying and block verification is shown in Fig. 13. In this framework, the authors deploy a series of Access Point (APs) to incentivize the transaction relay and block verification using DPoS-Based Lightweight Block Verification Scheme. Therefore, an important problem for AP is that how to improve the efficiency of transaction relaying and DPoS based block verification, while the payment (cost) is small.

In this work, a two-stage contract theory based on joint optimization scheme is proposed, where the AP serves as an employer who designs all kinds of contracts, pays rewards for employees, relays devices, and verifiers serve as employees. Relay devices should consider their battery energy, resource of occupied bandwidth, etc., and verifiers should consider their CPU cycles, energy consumption, etc. In order to maximize the expected utility of AP while satisfying the individual rationality and the incentive compatible constraints for transaction relaying and block verification, the objective function in the optimization problem is formulated as

where the limiting conditions $\{a,b,c,d,e\}$ are respectively expressed as

• (a)

  For each relay device, the reward paid from AP should be not less than its cost due to the rationality.

• (b)

  Similarly, for each verifier, the reward paid from AP should be not less than its cost due to the rationality.

• (c)

  AP needs to design the contract for the relay device or verifier flexibly according to their corresponding types.

• (d)

  Similarly, AP needs design the contract for the verifier flexibly according to their corresponding types.

• (e)

  For AP, the reward paid to relay devices and verifiers must be reasonable following a limitation.

where $V_{R}$ is the whole value of transaction relaying created by all the relay devices, and $R_{R}$ is the whole payment from AP to relay devices. Similarly, $V_{V}$ is the whole value of block verification created by all the verifiers, and $R_{V}$ is the whole payment from AP to verifiers.

However, to solve the optimization problem in a practical system, the constraint conditions often are set to non-convex, thus (5.2) cannot be solved directly.

By reducing some constraints, this optimization problem can be transferred into a convex problem. Finally, the optimal solution is the best strategy for transaction relaying and block verification, which can maximize the utility of AP while incentivizing the relay devices and block verifiers to accomplish their tasks optimally.

For blockchain system, the mathematical optimization tool is usually introduced to find the best pool selection strategy for miners to determine the best task offloading strategy, and to allocate resource for consensus process. Although existing researches show that the optimization can achieve the better system performance and security, some problems should be further studied in the future.

1. 1.

   The optimization problem and corresponding solution are usually for a specific situation in static or semi-static state. However, the practical blockchain system is dynamic and stochastic with some important parameters and constraints that are uncertain and might be changed over time and space. Therefore, these uncertainties should be further considered in optimization formulation.

2. 2.

   Due to the complexity of the blockchain system, the solution to the formulated optimization problem might be challenging. For example, the original problem should be transformed into a standard convex problem, decompose into multi-subproblem, or design an iterative approach to achieve a sub-optimal solution instead of the optimal one. Therefore, the solution method is another issue that should be addressed considering the solution quality, convergence speed and overhead.

3. 3.

   For modelling, some basic assumptions and simplifications with some typical mathematical properties are necessary especially for problem formulation and analysis, but they cannot describe the actual blockchain system accurately. Therefore, the acceptable gap between the reality (practical system) and ideality (mathematical model) should be also well studied.

In recent years, machine learning has been widely used in pattern recognition [110], data mining [111], etc., due to its capabilities in data management, analysis, and decision-making. In blockchain, machine learning can provide an efficient and intelligent approach to discover the malicious action and recognize the attacks to guarantee the data reliability, system security and user privacy. On the other hand, machine learning is also used to make the decision for resource allocation, block size setting and transactions scheduling to optimize the performance of blockchain system.

Malicious attackers can launch double spend attacking [112], denial of service attacking, and eclipse attacking on the blockchain network, which will cause the deteriorated security risk. In recent years, machine learning is introduced in blockchain to solve security problems. By using unsupervised/supervised learning algorithms such as the K-means algorithm and supervised SVM, etc., we can monitor the transaction in consensus process and the behavior of blockchain users [113]. Accordingly, as shown in Fig. 14, we can learn the characteristic of both honest and malicious actions, identify suspicious transaction, and malicious attacker or illegal activity in the network, which can reduce the possibility of successful attacks.

Nowadays, the resource consumption is an barrier for blockchain applications, especially in the case which is resource-limited such as the IoT device in the wireless network [14]. Therefore, it is necessary to consider the energy-saving in mechanism design and resource allocation. To develop an optimal strategy, we can define the state space $s(t)$, the action space $a(t)$, and the reward function $r(t)$ to represent the agent, environment, and the feedback between them in a machine learning manner [114]. The interaction among $s(t)$, $a(t)$ and $r(t)$ is shown in Fig. 15. In a real environmental transformation, the probability of going to the next state $s(t+1)$, is related to both the current state $s(t)$ and the previous state $s(t-1)$, occasionally related to even earlier state $s(t-2)$, so that the environmental transformation model may be too complex to model. Therefore, the way to simplify the environmental transformation model of reinforcement learning is to assume the markov property of state transformation: the probability of transformation to the next state $s(t+1)$ is only related to the current state $s(t)$, and has nothing to do with the previous state [115]. Accordingly, to maximize the long-term reward, the optimal strategy determination considering the interaction and feedback over time can be treated as a Markov decision process, and it is able to be achieved using reinforcement learning or deep reinforcement learning algorithms.

In addition, blockchain is a potential solution to the problems in machine learning. Most typically, the decentralized feature of blockchain can be used to solve the problem of single point of failure caused by aggregating machine learning models using centralized servers.

In this section, the previous work [116] is used as an example to illustrate the benefits of the combination of blockchain and machine learning.

Federated learning (FL) [117], as a promising training paradigm, was proposed by Google to tackle the privacy and security problems of centralized machine learning and to alleviate the communication load of the core network. As shown in Fig. 16 (a), many devices collaborate in solving a machine learning problem by updating the local model with their own local data, under the coordination of the centralized FL server. However, the traditional FL, which relies on a centralized FL server for model aggregation is vulnerable to experience service paralysis even when a single point of failure happens. All local models updated from devices will be distorted by the inaccurate global model aggregated at the FL server. In addition, for some devices with massive amounts of data, if there is no credible incentive mechanism, they are usually unwilling to participate in training, which brings great challenge to the rapid convergence of the FL model.

To address the problems mentioned above, H. Kim proposed a blockchained FL (BlockFL) architecture, which is shown in Fig. 16(b). With the distributed and non-tamperable characteristics of the blockchain, the use of a blockchain network instead of the centralized FL server can effectively overcome the issue of single point failure. The model parameters uploaded by the devices are taken as the transactions, and will be recorded in the candidate block after being verified by the associated miner. After all miners reached a consensus, devices can obtain the latest global model by aggregating the local model updates contained in the newly generated block downloaded from the associated miner. Generally, the global model is aggregated anywhere as long as the latest block can be obtained. In addition, the data reward for devices will be issued by the associated miner according to the size of the device data sample; the mining reward for miners will be issued by the blockchain network according to the total volume of data used by the connected devices. With the reasonable incentives, the blockchain-based FL can be positively driven for efficient training.

Due to the decentralized architecture of BlockFL, the malfunction of each miner only distorts the global model of its own devices instead of paralyzing the entire system. Moreover, such distortion can be recovered by interaction with other regular miners or federating with other devices associated with regular miners. Besides, by optimizing the block generation rate, the time for BlockFL to complete the model training can be reduced and the performance of the system can be improved.

In the literature, a number of works have shown that using machine learning in both data management and analysis can effectively monitor and classify the behavior of blockchain users, as well as recognize malicious behavior, suspicious user and illegal activity in the system, therefore both reduce the possibility of attacks and optimize the performance of system. Comparing with the traditional method of optimization or game theory, machine learning can adjust its strategy according to the changing of environment based on the long-term reward maximizing. However, there are still two existing problems that should be further investigated.

1. 1.

   Machine learning is valuable to understand the blockchain process and behavior in order to optimize system throughput and security (i.e., the transaction processing speed and malicious attack recognition), how to use the ability of machine learning in management, analysis and prediction to provide an intelligent guideline for decision-making and mechanism design is still an open issue.

2. 2.

   The impact of convergence and accuracy of selected machine learning algorithms on the performance and security of blockchain system should be considered.

Cryptography is the basic theory in blockchain, in which hashing, asymmetric encryption, timestamp, Merkle tree, etc. have been widely used to guarantee the security of transactions and the privacy of user information. The basic structure of blockchain is chain-of-blocks, the chain can be seemed as the relationship of blocks, and the block is an encapsulated data structure to cache data/transactions. In order to maintain the blockchain system, blockchain users perform consensus mechanism to generate blocks continuously, and hash algorithm in cryptography is carried out for transaction integrity, proof of consensus, writing new block in chain, etc. Besides, a binary Merkle tree is used for data structure which can confirm the existence and integrity of the transactions quickly.

Fig. 17 shows the application of asymmetric encryption in the blockchain. with asymmetric encryption, blockchain users can use the public key to encrypt information in transaction to ensure the security. Meanwhile, the private key is used to sign the transaction digitally by any blockchain user, and the others can use the corresponding public key to verify and to prevent in a distributed manner. For the privacy issue caused by data stored in the blockchain, some scheme can be adopted to provide anonymity decoupling users’ information from published transactions, involving of public key encryption scheme with keyword searchable, anonymous digital certificate publishing scheme, and zero-knowledge proof. For data processing security in the blockchain, homomorphic encryption technology enables users to encrypt the transaction data using the corresponding encryption algorithm before submitting the transaction data to the block chain network. The data exists in ciphertext, which will not reveal any privacy information of users even if it is obtained by the attackers. Meanwhile, the ciphertext operation result is consistent with the plaintext operation result.

In the meantime, we can apply cryptography theory for access control and authentication in the blockchain systems. As shown in Fig. 18, by controlling the right of data reading and user access, only the users who have the right attributes can decrypted the encrypted data and access the blockchain systems, while those users who do not have the right attribute cannot decypt data successfully. Thesefore, the security of storing data and information in the blockchain can be guaranteed by using crytography.

In this section,we use the previous work [138] to introduce the application of cryptography in blockchain. Due to the attributes of blockchain, many researchers try to apply blockchain technology to vehicular networks to improve its security. However, possible privacy leakage issues will reduce the enthusiasm of vehicles to participate in ad dissemination. In order to solve the vehicle privacy problem of the blockchain-based vehicular networks, the authors use Zero-knowledge proof of knowledge (ZKPoK) to propose a new blockchain-based ad dissemination framework which is shown in Fig. 19.

To ensure the validation of ad dissemination and improve the security of the vehicular networks, this work designs a concrete, fair and anonymous scheme under the proposed framework. In order to ensure fairness, Merkle hash trees and smart contracts are used to implement the proof of ad reception to mitigate “free-riding” attacks. In addition,the proposed scheme can protect vehicles’ privacy in terms of anonymity and conditional unlinkability based on zero-knowledge proof techniques. In addition, we briefly explain the five-stage process involved in the privacy protection blockchain architecture using Merkle hash tree and ZKPoK.

• •

  System Setup: Register authority initializes the system and generates public and private keys based on predefined security parameters.

• •

  Registration: Advertisers and vehicles run sub-protocols to obtain valid certificates or keys individually. In the sub-protocol for advertisers, the registration authority adds registration information based on the ECDSA public key sent by the advertiser. In the sub-protocol for vehicle operation, register authority verifies the vehicle’s identity information according to the ZKPoK sent by the vehicle.

• •

  Ad Publication: Advertisers with valid certificates package ad information as a transaction and submit it to smart contracts, where the ad information is stored as the form of Merkle hash trees. Advertisers also generate ECDSA signatures so that participants can verify the validity of the transaction. Then, they select the suitable roadside unit to broadcast the ad.

• •

  Ad Dissemination: The selected Road Side Unit (RSU) broadcasts ad to nearby vehicles, or vehicles forward advertising to other vehicles. In addition, both the forwarding vehicle and the receiving vehicle will use ZKPoK to generate anonymous credentials to prove that the forwarding process is actually took place.

• •

  Reward Payment:. At this stage, RSU checks the validity of the anonymous credential and proof-of-ad-receiving submitted by the forwarding vehicle. If the verification is passed, the valid advertisement receiver and sender will receive corresponding rewards.

Cryptography is the key element for confidentiality, integrity, authentication and non-repudiation for security and privacy in blockchain using the typical scheme like digital signature, asymmetric encryption, hashing and etc. These methods focus on security and privacy protection in an external manner, which are to design a powerful wall around the blockchain system to deny the attacks from malicious users. However, the system would be vulnerable if any bugs found by the attacker to break. Therefore, how to empower the security and privacy ability of blockchain system in a systematic manner providing inherent safety is still an open issue. Moreover, the mechanism and protocol should be designed to optimize the performance as well as security simultaneously.

From the resource consumption perspective, many blockchain protocols, which consumes less power than PoW, have been proposed, such as PoS, IOTA and so on. However, they struggle to be widely used in wireless networks because of their scalability. As to another type of blockchain protocols which vote instead of calculating, such as PBFT, Hashgraph and so on, they are still not widely applicable to wireless networks due to their communication complexity. From the communication perspective, caused by blockchain’s transaction release, consensus interaction, ledger updating and so on, additional communication overhead is bound to be incurred along with the improvement of data security brought by blockchain. There have to be a tradeoff between the communication performance and the security performance of the blockchain network. In conclusion, essential questions to be technologically addressed include: $\left.{\rm{1}}\right)$ how to design a blockchain protocol with high scalability and appropriate power consumption, communication complexity while ensuring its security, $\left.{\rm{2}}\right)$ how to compromise between blockchain’s performance and network’s performance.

In addition to the technical matters, the lack of killer applications [155] and business models also hinders the large-scale application of blockchain. For private or consortium blockchains, only the untamperability of the data on chain can be guaranteed. While the authenticity of the data off chain requires the cooperation of other technologies form Internet of Things. In terms of public chain, it is more about the transformation of the whole system, which is suitable for the system without effective incentive system or reliable allocation mechanism. The first areas to to be implemented will be those that have formed a consensus but lack incentives and cannot be implemented on a large scale.

The lack of policies, regulations, standardization and other related policy issues are hindering the mass commercialization of blockchain. In the formulation of them, the development and limitations of blockchain should be taken into consideration. At the mean time, the selection of solutions in blockchain must also comply with the restrictions of policies, regulations and standards.

Blockchain is built in a P2P network, which means that a large amount of communication overhead will be added in terms of network traffic and system processing capacity. Because current applications are highly dynamic, data sources frequently change. This indicates that the node may have to send a large number of update transactions, which will further increase the communication overhead. On the other hand, blockchain deployment in wireless is foreseeable in the near future [36]. In this case, blockchain services rely on wireless communication networks to reach consensus. During the consensus process, blockchain nodes are connected through wireless channels. However, due to factors such as wireless channel fading and unauthorized malicious interference, the wireless connection among blockchain nodes may be attacked, and the uplink or downlink transmission may fail, thereby reducing the probability of successful transactions. Therefore, in the wireless blockchain system, a framework is needed to measure the communication overhead and communication quality in the communication process.