FEDQ-Trust: Efficient Data-Driven Trust Prediction for Mobile Edge-Based IoT Systems

In the realm of model-driven IoT trust prediction, [19] introduces a trust assessment method based on crowdsourcing. [20] presents an algorithm combining WMA-OWA for trust evaluation using five metrics in extensive P2P networks. A broader approach to trust assessment in distributed networks is offered by [1], defining network trust and creating entropy and probability-based models for it. [21] investigates three challenges in trust evaluation within VANETs, suggesting an indirect trust evaluation method using reinforcement learning and three trust attributes. Direct and indirect node trust assessment methods, incorporating six trust metrics, are proposed by [22]. [23] delivers a trust model for IoT systems, applying fuzzy theory and exploring the concepts of trust and reputation within IoT contexts. Both [24] and [25] introduce a dynamic trust management protocol to address malicious behaviours in IoT systems, distinguishing between malicious and non-cooperative nodes and setting stringent criteria. Enhancements to these protocols, including storage management strategies for scalability, are made in [26]. [27] proposes a sophisticated, layered architectural design aimed at crafting a trust management system for IoT. [28] describes a trust and privacy-preserving recommendation scheme for vehicle platoons, involving trust valuation for selecting the lead vehicle. [29] introduces a framework for assessing IoT service trustworthiness, adapting to consumer usage patterns and employing a novel algorithm to detect service-specific trust indicators.

The major limitation of model-driven methods lies in their specific applicability to certain IoT services or groups, hindered by the diverse characteristics of different IoT services. These services often exhibit unique, context-specific features that vary significantly, challenging the effectiveness of a single model across all scenarios. These model-driven approaches necessitate manual analysis of trust characteristics by domain experts for each distinct IoT service, thereby limiting their flexibility and struggling with accuracy and efficiency in diverse IoT environments.

In the field of centralized IoT service trust prediction, a trust management framework using Support Vector Machine (SVM) was proposed in [30]. It focuses on trust management communication. [31] considers trust assessment as a classification problem using non-probabilistic binary SVM classifiers, with an emphasis on social network trust evaluation. EMLTrust [32] is a reputation system for Mobile Ad-hoc Networks (MANETs) based on digital signatures, utilizing SVM. Additionally, [6] and [4] leverage machine learning for device assessment in IoT networks and single-agent trust quantification in multi-agent systems, respectively.

Centralized methods have significant limitations in IoT service trust prediction, including inadequate high-dimensional information processing, lack of direct communication between nodes in MEC environments, and over-reliance on core networks. These limitations lead to suboptimal performance in large-scale and dynamic settings [7].

Several studies focus on developing advanced methodologies for data-driven IoT service trust prediction. [33] proposes a probabilistic graphical model for sensor trustworthiness prediction in IoT and suggests an algorithm to optimize energy use in sensor networks. [34] devises a social network trust framework using an MLP with echo propagation. Additionally, [13] defines a distributed trust prediction model for MEC environments as a Network Lasso problem, using Alternate Direction Method of Multipliers (ADMM) for SVM in model training and credibility prediction. This is further refined by [7] with the Stochastic ADMM (S-ADMM) method to improve its scalability. Furthermore, [35] introduces a neural network framework for evaluating service provider trust in crowdsourced IoT. Meanwhile, a deep federated learning method is proposed for enhancing IoT security to identify and manage malicious nodes effectively in [14]. The study in [17] proposes to employ FedEM for predicting the trustworthiness of IoT services in MEC environments. It addresses the challenge of statistical heterogeneity in trust modeling.

Current IoT trust prediction solutions do not fully address the issue of statistical heterogeneity in distributed MEC environments. For example, the pioneering framework introduced by [7] has several areas needing enhancement. Specifically, our experiments reveal that the framework’s prediction accuracy is suboptimal. In addition, it necessitates numerous communication rounds and extended time for convergence. The study conducted in [14] focuses on dealing with the heterogeneity of data in its analysis. It lacks a detailed exploration of the heterogeneity of mixture distribution. Additionally, there is an absence of discussion regarding scenarios where IoT environments are constrained by limited resources. The statistical heterogeneity arising from mixture distributions has been effectively addressed in [17]. However, it does not fully mitigate the challenge for efficient training of trust models in the MEC environment. This is crucial for rapid service selection decisions and service delivery in IoT systems.

The Federated Expectation-Maximization (FedEM) is a unique method within federated multi-task learning. It leverages multi-task learning to understand correlations across different clients, enlarging the node sample size and boosting performance. FedEM assumes each local data distribution of a client is a mix of foundational distributions. It tackles statistical heterogeneity in federated learning by learning shared component models and personalized mixture weights. This approach adapts to data diversity, maintaining high accuracy and fairness while addressing distributional differences [18].

FedEM is based on two main premises: Premise 1: Each local data distribution $\mathcal{D}_{t}$ is a blend of $M$ latent distributions $\tilde{\mathcal{D}}_{m},1\leq m\leq M$, illustrated by the following equation:

Premise 2: For all $m\in\left[M\right]$, $p_{m}\left(\mathbf{x},y\right)=p_{m}\left(\mathbf{x}\right)$.

The goal of FedEM is to find the optimal parameters of the components $\Theta^{*}=\left(\theta_{m}^{*}\right)_{1\leq m\leq M}$ and mixture weights $\Pi^{*}=\left(\pi_{t}^{*}\right)_{1\leq t\leq T}$ by minimizing the negative log-likelihood $f\left(\Theta,\Pi\right)$. This complex problem is typically tackled using the Expectation-Maximization (EM) algorithm, which consists of two steps:

- E-step: The algorithm computes the probability of each data point belonging to each component by updating the latent variable distribution $q_{t}$ for every data point, as follows:

- M-step: The parameters $\{\Theta,\Pi\}$ are updated by maximizing the expected log-likelihood. This includes identifying the optimal component parameters and mixture weights using standard optimization techniques.

With these steps, the algorithm iteratively refines the estimated parameters, driving the model towards the optimal solution.

The objective of federated optimization can be interpreted in FedEM as the minimization of the global objective function, namely the negative log-likelihood function $f(\Theta,\Pi)$. This function consists of a weighted average of the local loss functions of all MEC environments. The global optimization problem can be written as follows:

where $f(\Theta,\Pi)$ is the global objective function, which is the weighted average of the local loss functions of all MEC environments, which can be expressed as:

where $f(\Theta,\Pi)$ is the global negative log-likelihood across all MEC environments. $n$ and $n_{t}$ are the total and per-environment data points respectively. $s_{t}$ denotes data from the $t$-th environment and $\pi_{t}$ its mixing weight. $p(s_{t}|\Theta,\pi_{t})$ represents the probability of data $s_{t}$ given the global parameters $\Theta$ and $\pi_{t}$.

Deep Q Network (DQN) is a Q-Learning variant that utilizes deep learning for function approximation. The key to DQN lies in its ability to make intelligent decisions. It autonomously learns and identifies the optimal strategies in complex environments, enabling effective decision-making across various application scenarios [36]. In the context of federated learning, DQN can evaluate and select clients that bring higher efficiency to model training. This selection is based on the relationship between model weights and a reward function focused on increasing validation accuracy and reducing communication rounds. This capability allows DQN to effectively choose clients that contribute most significantly to the overall learning process. In this regard, it is capable of optimizing training time and resource utilization [16].

The Q-Learning algorithm can be represented as:

Here, $(s,a)$ is the current state-action pair, $r$ represents the immediate reward, $(s^{\prime},a^{\prime})$ are the next state and action, and $\gamma$ is the discount factor.

However, the complexity of learning the optimal action-value function, $Q^{*}(s,a)$, increases with high-dimensional spaces or continuous states/actions. DQN overcomes this by using a deep neural network, $Q(s,a;\theta)$, approximating the $Q^{*}(s,a)$ function. This network’s parameters, $\theta$, are learned by minimizing a series of loss functions, $L_{i}(\theta_{i})$:

The target value, $y_{i}$, is defined as:

DQN employs Experience Replay to disrupt continuous sample correlation, thus making the training more stable and efficient. The agent’s experiences are stored in a replay buffer, $\mathcal{M}=\{e_{1},...,e_{t}\}$, and minibatches of these experiences are sampled during the learning process to update the network parameters.

Our proposed solution, FEDQ-Trust, leverages a federated multi-task learning algorithm for trust prediction in MEC-based IoT systems. We integrate this algorithm with a DQN model to enhance efficiency and performance. The core concept of FEDQ-Trust is to address trust prediction as a federated optimization problem aimed at minimizing the global negative log-likelihood (Equation 11).

FEDQ-Trust operates on the premise that the trust prediction model of each MEC environment should be trained on its local data distribution to provide accurate predictions while preserving the privacy of local data. This recognition arises from the fact that IoT service trust data in different MEC environments may exhibit distinct data distributions. The integration of DQN into our solution facilitates efficient environment selection during training, optimizing the overall training process and accelerating convergence.

Our proposed algorithm is implemented within a distributed architecture involving decentralized MEC environments and a central cloud. In this architecture, the local trust prediction parameters of the model within each MEC environment are transmitted to the central cloud through a bidirectional communication link. This enables federated learning, where local model parameters are aggregated in the central cloud to create a global trust prediction model capable of adapting to variations among MEC environments.

Each MEC environment performs two critical tasks: EM updates for component weights optimization and local model training on locally aggregated data distributions. This distributed federated learning strategy consolidates data from diverse MEC environments in the central cloud, resulting in more accurate models. Our data-driven approach enhances trust prediction accuracy and can be applied across various distributed IoT systems in MEC environments. In summary, our algorithm achieves distributed trust prediction by enabling two-way communication between the global cloud and each MEC environment, leading to more precise and reliable trust predictions.

The algorithm follows a workflow consisting of six key steps, as depicted in Figure 2, outlining the federated optimization process:

• 1)

  Initialization: The initial step broadcasts the weights of the global trust prediction model from the central cloud to each MEC environment and the initialization of the local mixture weights by each MEC environment. Furthermore, the local mixture weights initialization accommodates data distribution of each respective MEC environment, influencing the upcoming model training and parameter updates.

• 2)

  EM Updating and Local Training: In the second step, we require each MEC environment to perform an update on their locally optimized component weights, a process carried out through the Expectation-Maximization (EM) step. Concurrently, every MEC environment will employ its locally optimized component weights and local datasets for model training, followed by the transmission of these trained model weights back to the central cloud for the next round of aggregation. It is during this step that each MEC environment implements updates on their local mixture weights through the EM step (See Equations 8-10), thereby tailoring it to better fit the local dataset.

• 3)

  Local Model Uploading: The third step involves each MEC environment uploading its locally trained model weights to the central cloud for model aggregation.

• 4)

  DQN-Based Environment Selection: The fourth step introduces a DQN model to select the optimal MEC environment. The DQN model is responsible for deciding which MEC environments will receive the updated global model weights in the following step, improving the efficiency of the federated learning process.

• 5)

  Aggregation: In the fifth step, after the central cloud receives the model weights of the local MEC environment, the central cloud aggregates them according to the model weights of each MEC environment, thus completing a round of communication. In this step, the central cloud will receive model weights from multiple MEC environments and aggregate them based on the weights of each MEC environment. This helps synthesize information from multiple environments and derive parameters for a global model (see Equation 14).

• 6)

  Local Model Updating: For the final step, only the MEC environments selected by the DQN model in step four receive the aggregated global weights. These environments simultaneously update their local mixture weights and local model weights before sending this updated data back to the central cloud. This marks the beginning of the next iteration, which will continue until optimal model parameters are obtained, or a predetermined stopping condition is reached.

Our study addresses the issue of heterogeneity in MEC-based IoT systems. This heterogeneity results from different data distributions and non-identically distributed (non-IID) data. We use the FedEM algorithm to manage this challenge [18]. The FedEM algorithm is known for dealing effectively with non-IID and mixed data in federated learning. It performs Expectation-Maximization (EM) steps in each MEC environment. These steps help in federated optimization. They also improve prediction abilities in various environments. Section 3.1 provides more details about the FedEM algorithm.

Our framework uses a multi-layer perceptron (MLP) as its basic learning model. The MLP has an input layer, a hidden layer with many perceptrons using ReLU functions, and an output layer. In the output layer, a sigmoid function activates one perceptron. Each MEC environment trains its local MLP as a binary classifier. The performance of these classifiers differs because of the various data distributions they face. However, we use federated optimization to combine them into a global model for trust prediction.

Our methodology further refines the efficiency of convergence of FedEM by integrating a reinforcement learning approach powered by a DQN agent. This DQN agent intelligently selects a specific subset of MEC environments in each iteration of federated learning, optimizing the convergence speed while maintaining model accuracy. The parameters of the global model constitute the state of DQN, which we simplify using Principal Component Analysis (PCA) due to the inherent high dimensionality of parameters in deep learning applications [16]. Within the FedEM framework, individual models are tailored to each distribution, followed by PCA application to condense the model parameters into a manageable state space. The DQN is rewarded based on the accuracy of the current iteration and a predefined accuracy. The technical details of DQN are explained in Section 3.2.

Our DQN-enhanced FedEM strategy is outlined in Algorithm 1. First, the central cloud server prepares the replay memory and sets the initial model parameters. Then, each MEC environment starts its own setup, as shown in lines 1-5 of the algorithm. Before the main training begins, we run an initial FedEM training cycle. This first cycle sets the starting state of the DQN model, as mentioned in line 6. After this, we apply PCA to the parameters from both global and local models that finished the initial FedEM training. This process simplifies the state space, as described in lines 7-10 of the algorithm.

The global model’s weights help choose an action from the MEC environment pool once DQN training begins. This selection process is in line 12 of the algorithm. After this choice, we update the model parameters and send them to the selected MEC environments. This starts a new FedEM training cycle, shown in line 13. Then, these local environments train with their data. The central cloud combines these updated local models, as line 14 indicates. Next, we apply PCA to the new global model. This compresses its parameters, forming the next state for the DQN, as lines 15-16 describe. We calculate the reward by comparing the target and current accuracy. In our tests, we set the accuracy threshold $\Xi$ at 64, as line 17 states.

The tuple that includes the current state, the action taken, the resulting reward, and the next state is saved in the replay memory, as line 18 of the algorithm shows. The algorithm continues by choosing random transition samples from the replay memory for the DQN network’s training phase. This is line 19 of the algorithm. Line 20 deals with how to calculate the target value $y_{j}$. If the observed accuracy $A_{t}$ meets or exceeds the set accuracy goal $A$, then $y_{j}$ equals the current reward $r_{j}$. If $A_{t}$ is below $A$, $y_{j}$ is the sum of $r_{j}$ and the decay factor $\gamma$ multiplied by the highest forecasted Q value for all possible actions $a^{\prime}$ in the next state $s_{j+1}$. Line 21 carries out an optimization step. It uses gradient descent to reduce the difference between the expected Q value $y_{j}$ and the Q value $Q(s,a)$ predicted by the network. This matches the loss function in Equation (8). The procedure in line 11 repeats until the algorithm completes the set number of training cycles. It ends with the final model parameters for prediction.

Our algorithm uses a process called ”EMStepWithAggregate” during federated optimization. It works at the same time across the chosen $T$ subsets of MEC environments. This process carries out Expectation (E) and Maximization (M) steps for each part of the mixture distribution. In the E-step, the algorithm updates the posterior probabilities for hidden aspects of each data sample. This is detailed in line 4 of the EM algorithm and follows the updating rule in Equation (2). In the M-step, it changes the weights and parameters of the mixture components. This is based on lines 6 to 7 by following the formulas in Equations (3) and (4). Each MEC environment sends its new parameters to the central cloud after updating. This is shown in line 9 of the EM algorithm. The central cloud then combines these parameters. It does a proportional aggregation as described in lines 11 to 13 of the EM algorithm. This combination creates the global model parameters.

A series of experiments were conducted to thoroughly assess the effectiveness of the proposed method in predicting the trustworthiness of IoT services in MEC-based IoT systems. The conducted experiments are elaborated as follows.

• •

  Accuracy stands as a key performance indicator for any machine learning approach. This metric is particularly essential, as it quantifies the percentage of samples within MEC scenarios that are accurately forecasted by the algorithm. This metric serves as a benchmark to evaluate the comparative effectiveness of various models in predicting the trustworthiness of IoT services.

• •

  Training Time Comparison provides insight into the computational speed of our method relative to others. This metric represents the complete duration required for a model to achieve convergence in its training phase. By evaluating the elapsed time across different models, we can determine which one operates most efficiently. This is especially vital in extensive MEC environments, where the speed of computation plays a major role in overall performance.

• •

  Convergence Iterations Comparison serves as a crucial metric. It measures data exchange during model training. The metric counts interactions between the central server and local devices. These interactions continue until the model converges. The measure identifies models with optimal performance and minimal communication rounds. It also shows how much the DQN component reduces computational complexity. This factor is important in MEC environments with limited communication resources.

• •

  Comparison of DQN Component and Random Selection focuses on two key aspects. First, it looks at how intelligent MEC environment selection methods with DQN components differ from traditional random selection methods in resource selection efficiency. Second, it aims to validate the effectiveness of the DQN component. The contrast with random selection highlights the advantages of the DQN component.

The following models are selected as the baseline for comparison.

FedTrust [14] is a federated learning-based IoT service trust prediction approach. It excels in identifying harmful nodes in MEC-like environments using a trust dataset enriched with knowledge, experience, and reputation.

S-ADMM [7] is another state-of-the-art data-driven IoT service trust prediction approach. It can transform the distributed trust prediction problem into a network lasso problem.

Local SVM [13] conducts IoT service trust prediction independently in each MEC environment.

The following real-world IoT security datasets are leveraged for our evaluation.

UNSW-NB15 Dataset²²2https://research.unsw.edu.au/projects/unsw-nb15-dataset: This dataset comprises an extensive range of network traffic data. It includes representations of both standard network operations and artificial attack activity, which were synthesized in a controlled lab environment. The dataset encompasses approximately 2 million entries, each detailed with 49 distinct features. Every entry aligns with an outcome, categorized as either a normal transaction or one among nine varied attack types. In our experimental context, these nine attack types were classified as malicious, while all standard transactions were labelled as non-malicious.

N-BaIoT Dataset³³3https://archive.ics.uci.edu/ml/datasets/detection_of_IoT_botnet_
attacks_N_BaIoT: This dataset is an aggregation of network traffic data, sourced from compromised real commercial IoT devices. It features roughly 7 million entries, with each entry detailed by a comprehensive set of 115 features. In this dataset, attack types are broadly divided into two main categories, each encompassing various specific types of attacks. All instances of these attacks are labelled as malicious, whereas the rest of the data, representing normal operations, are identified as benign.

We allocated 80% of the data for training and 20% for testing to maintain a consistent and unbiased evaluation of the models. We divided the datasets into 100 local datasets of MEC environments. The sizes of these local datasets differ considerably, showcasing a range of data distributions that mirror the diverse nature typically seen in real-world MEC systems. The training dataset sizes range from 5 to 613,791 entries, while the test dataset sizes span from 19 to 215,016 entries. In federated learning with artificial neural network (ANN) models, a trust value of 0 means benign instance, and 1 means malicious instance. This differs from other models that use 1 for benign instances and -1 for malicious instances.

We systematically segmented the dataset and generated a mixture distribution before model training. This process was crucial to ensure uniformity across all models. This approach guarantees that our experimental dataset aligns with the mixture distribution hypothesis. Initially, the dataset is segmented into two distinct clusters. Within each cluster, the samples are then distributed across 100 MEC environments, guided by a Dirichlet Distribution with an alpha value of 0.4⁴⁴4 The work in [17] adopts an alpha value of 0.3, resulting in no positive examples in a few local MEC environments. Consequently, the experimental results are different from those presented in [17]. . For each MEC environment, sample selection from each cluster is executed in a manner that aligns with the Dirichlet distribution of the corresponding cluster. Specifically, the Dirichlet distribution is used as a prior for the multinomial distribution, and Bayesian inference updates the posterior distribution with each new sample. The update forms a new probability distribution. This results in a mixture distribution. It shows the diverse characteristics of the data in MEC environments [18].

We assess the efficacy of our FEDQ-Trust model. We compare it with the baseline models depicted in Section 6.2 with the following Key Performance Indicators (KPIs).

Accuracy stands as a pivotal measure in the realm of machine learning, holding particular significance in our study. This metric quantifies the ratio of correctly predicted instances in the MEC environments relative to the model’s predictions. Accuracy is the number of correctly predicted instances divided by the total number of samples.

True Positive Rate (TPR), also known as sensitivity, is very important in our analysis. This is because our datasets have an imbalance of positive and negative samples. TPR evaluates the proportion of actual positive instances (true positives) accurately identified by the model from all actual positive instances present. This rate is determined by dividing the number of true positive outcomes by the total of true positives plus false negatives.

False Positive Rate (FPR), also known as the probability of false alarm, assumes a significant role in our study. This importance comes from the uneven distribution of positive and negative samples in our datasets. FPR measures how often the model wrongly identifies negative instances as positive. It compares these cases to all actual negative instances. This metric is calculated by dividing the number of false positives by the sum of false positives and true negatives.

Communication Rounds serves as a key metric to gauge the data exchange load involved in the training of models. This count reflects the number of interactions required between the central server and local devices for a model to reach convergence.

Elapsed Training Time is a important KPI. It measures the training efficiency of our algorithm. This is important for comparing it with other models. This metric is defined as the total time a model requires to reach convergence throughout the training phase.