DAMe: Personalized Federated Social Event Detection with Dual Aggregation Mechanism

Social event detection, aiming to identify potential social events from social streams, is a longstanding and challenging task. Recent SED methods primarily rely on Graph Neural Networks (GNNs) (Peng et al., 2021, 2019; Cao et al., 2021; Cui et al., 2021; Ren et al., 2021, 2022a; Wei et al., 2023; Ren et al., 2023). These approaches construct message graphs to represent social message data, integrating various attributes that complement each other and serve independent roles in propagating and aggregating text semantics. For instance, KPGNN (Cao et al., 2021) builds an event message graph using user, keyword, and entity attributes, then employs inductive Graph Attention Networks (GAT) to learn message representations. Furthermore, some approaches adopt multi-view learning strategies to enhance the feature learning process. MVGAN (Cui et al., 2021), for instance, learns message features from both semantic and temporal views and incorporates an attention mechanism to fuse them. ETGNN (Ren et al., 2023) focuses on learning representations from co-hashtag, co-entity, and co-user views. However, current works have not yet explored methods utilizing federated learning to enhance the comprehensiveness and accuracy of SED.

Federated Learning (FL), an advanced paradigm for decentralized data training, has garnered significant attention in recent years (Zhang et al., 2021). FedAvg (McMahan et al., 2017) achieves collaborative learning across decentralized devices by locally training models and aggregating parameters on a central server. On that basis, FedProx (Li et al., 2020) introduces regularization that enhances model convergence and generalization by considering the smoothness of the global model in FL.

Due to the statistical heterogeneity in FL, a centralized global model may lower the performance of certain participants (Zhang et al., 2023). Hence, personalized FL has garnered significant attention (Tan et al., 2022). We categorize pFL methods into three types. Fine-Tuning-based Methods involve learning a global model, which clients then fine-tune on their respective sides to achieve personalization (Fallah et al., 2020; Collins et al., 2021; Yu et al., 2024). For instance, Per-FedAvg (Fallah et al., 2020) regards the global model as an initial shared model, allowing all clients to fine-tune it with local data to fit their specific needs. Personalized Layer/Model-based Methods offer flexibility in model architecture, allowing for variations such as sharing specific layers or training additional local models (T Dinh et al., 2020; Li et al., 2021a; Yang et al., 2023; Liu et al., 2023). For example, FedACK (Yang et al., 2023) employs GAN-based knowledge distillation for cross-model and cross-lingual social bot detection. Aggregation-based Methods achieve personalization through server centrally aggregates specialized global models for the participants or clients directly exchange parameters among themselves in a decentralized setting, allowing them to select the information they desire (Huang et al., 2021; Li et al., 2021b; Zhang et al., 2020; Luo and Wu, 2022; Sun et al., 2021; Chen et al., 2022; Zhang et al., 2023).

Previous studies have overlooked the possibility of servers providing a suitable model while allowing clients to integrate helpful knowledge. To this end, this work focuses on personalization to enhance local performance via global and local aggregation.

This paper considers FL with one central server and $K$ clients. The dataset $\mathcal{D}_{k}$, locally collected by client $k$, remains inaccessible to others. Below is an overview of the training process for the classic FL algorithm FedAvg (McMahan et al., 2017):

Step 1:

Initialization. At the initial communication round $r=0$, all local model parameters of $K$ clients are initialized as the global model parameter: $\theta_{0}^{l_{k}}\leftarrow\theta_{0}^{g}$, where $\theta_{0}^{l_{k}}$ and $\theta_{0}^{g}$ denotes the model parameters of client $k$ and the server at round $0$, respectively.

Step 2:

Client Update. Each client $k$ trains the model on their private dataset $\mathcal{D}_{k}$ with task objective $\mathcal{L}(\mathcal{D}_{k};\theta_{r}^{l_{k}})$. Then, upload the trained local model parameters $\theta_{r+1}^{l_{k}}$ to the server.

Step 3:

Server Execute. The server aggregates the received parameters by $\theta_{r+1}^{g}=\sum_{k=1}^{K}\frac{N_{k}}{N_{sum}}\theta_{r+1}^{l_{k}}$, where $N_{k}$ denotes the number of data samples of client $k$, and $N_{sum}$ is the total number of data samples across all clients. Then, the server distributes the new global model parameters to clients in the following round.

Iterate Steps 2 and 3 continuously until the final communication round. The global objective of the overall FL process is:

Given a collection of social messages $M$, a social event detection algorithm aims to learn a model $f(M;\theta)=E$ from $M$, where $\theta$ is the model parameter, $E=\{e_{j}\mid 1\leq j\leq|E|\}$ is the set of events (all labels).

Current federated paradigms are vulnerable to injection attacks (Lyu et al., 2022). In this work, the threat model is defined by the presence of malicious clients deliberately injecting backdoors within the training data (data poisoning attack) or uploading corrupted parameters to the server (model poisoning attack) to sabotage the FL process (e.g., performance, collaboration). Such attacks could have profound repercussions on the global model and threaten FL’s reliability and accuracy. We analyze the robustness of the proposed framework against injection attacks in Section 6.2.

For FedSED, we apply GAT (Graph Attention Network) as our backbone SED model and map the text encoding of different languages into a shared vector space.

We introduce a local aggregation mechanism, where clients learn a strategy that incorporates global knowledge while preserving their local characteristics rather than being directly overridden by the global model. The following optimization problem is formulated to describe the local aggregation process at the $r$-th communication round for client $k$:

where $\theta^{l_{k}}_{r}$ and $\theta^{g}_{r}$ denote the local and global model parameters, respectively. $\lambda_{r}^{k}\in\mathbb{R}$ represents the aggregation weight (the weight of local preservation). Local aggregation strives to determine the optimal or near-optimal weight $\lambda_{r}$ that allows clients to acquire the maximum amount of knowledge. The process of $\theta^{l_{k}}_{r+1}\leftarrow\widetilde{\theta}^{k}_{r}$ is described in Section 4.4.

Bayesian Optimization (BO) algorithm (Frazier, 2018) is a widely employed approach for optimizing functions with costly or challenging direct evaluations. We utilize BO for determining the aggregation weight $\lambda_{r}$ for Local Aggregation (BOLA), as shown in Figure 1(d). BOLA is accomplished through a three-step BO procedure: first defining the objective function, then modeling it using a Bayesian statistical model, and finally determining the subsequent sampling position by an acquisition function.

Under the federated framework described in Section 3.1, personalized global aggregation aims to provide clients with maximum external information by producing global models that can benefit individual clients more. The server needs an aggregation strategy that considers client heterogeneity and individual characteristics to maximize external knowledge for all clients. To achieve this objective, we construct a client graph $G_{client}$ based on clients’ similarity. By minimizing the two-dimensional Structural Entropy (2DSE) of $G_{client}$, a graph capturing the internal similarities among clients is obtained, finalizing the Global Aggregation strategy for each client (SEGA). This process is demonstrated in Figure 1(b).

$G_{client}$ is an undirected, fully connected, weighted graph consisting of $K$ nodes corresponding to $K$ clients, with their similarities as edge weights. The similarity between client models can be estimated by providing them with the same input and measuring the similarity between their respective outputs. On this basis, the server first generates a random graph $G_{random}$ as input to all client models (Holland et al., 1983). With graph pooling (Lee et al., 2019), the server obtains different client models’ representations of the same graph, and the similarity between client $u$ and $v$ is calculated as:

where $\tilde{h}_{u}$ is the averaged output of all node embeddings in the input graph $G_{random}$ and $\text{sim}(u,u)=1$.

Upon constructing the client graph $G_{client}=(V,E)$, we minimize the 2DSE of the graph, resulting in a partitioned graph, which serves as the basis for the aggregation strategy. Suppose $\mathcal{P}=\{X_{1},X_{2},\dots,X_{L}\}$ forms partitions of nodes in $V$, where $L\leq K$ is the total number of partitions, $V=\{c_{1},...,c_{K}\}$ represents the set of client nodes, and $X_{l}$ denotes the $l$-th partition that contains specific client node(s). The 2DSE of the client graph $G_{client}$ is calculated as follows (Cao et al., 2024; Li and Pan, 2016):

where $L$ denotes the number of total partitions, $n_{l}$ denotes the number of client nodes in partition $X_{l}$, $d_{i}^{(l)}$ denotes the degree of the $i$-th client node in $X_{l}$, $\operatorname{vol}(\cdot)$ computes the volume, and $g_{l}$ denotes the sum of degrees of edges with one endpoint in partition $X_{l}$. The objective of the minimization process is to assign each client node $c_{k}$ to a distinct partition $X_{l}$. Specifically, each client node is initially treated as an individual partition. New partitions are formed by iteratively merging different partitions. The changes in the 2DSE before and after merging are observed to identify the partitioning scheme that yields the lowest overall 2DSE and generates the desired partitions. We leverage the greedy strategy in (Li and Pan, 2016) to minimize 2DSE. The difference in 2DSE before and after merging $X_{i}$ and $X_{j}$ into $X_{l}$ is calculated as follows:

where the calculation of $H_{X_{l}}^{\mathcal{P}^{\prime}}(G_{client})$, $H_{X_{i}}^{\mathcal{P}}(G_{client})$, and $H_{X_{j}}^{\mathcal{P}}(G_{client})$ follows Equation 10, to compute the 2DSE of partition $X_{*}$ under the partion $\mathcal{P}^{*}$. $H^{\mathcal{P}^{\prime}}(G_{client})$ denotes the 2DSE of $G_{client}$ obtained by merging $X_{i}$ and $X_{j}$ into $X_{l}$. Note that we always merge the two partitions with the smallest $\Delta$SE until all $\Delta$SE $\geq 0$, thus obtaining the final partitions $\mathcal{P}_{final}=\{X_{1},...,X_{L}\}$. Based on the final partition, the global aggregation strategy aims to aggregate information within each partition. Specifically, in the $j$-th partition $X_{j}$, all client nodes are connected by edges weighted according to their similarities. For all nodes in the partition, the global model $\theta_{u}^{g}$ for client $u$ is obtained by:

where $v\in N(u)$ represents the node within the same partition as $u$, $\theta_{v}^{l}$ is the local model of client $v$, and $\alpha_{uv}$ is the normalized weight between client $u$ and client $v$, computed as:

In Section 4.2, we introduced a local aggregation strategy that aggregates $\theta^{g}_{r}$ and $\theta^{l}_{r}$ into $\widetilde{\theta}^{l}_{r}$. This section describes the local optimization of $f(\widetilde{\theta}^{l}_{r})$ with local data, maintaining the proximity between the local and the global models while preventing overfitting to the local data. The overall process is shown in Figure 1(e).

We conducted experiments on 6 datasets, spanning 6 languages and 2 platforms. Table 1 presents the statistics of all datasets. The English Twitter (McMinn et al., 2013) dataset, the French Twitter (Mazoyer et al., 2020) dataset, and the Arabic Twitter (Alharbi and Lee, 2021) dataset are publicly available. We collect the rest of the datasets from Weibo in Chinese and Twitter in Japanese and German. First, we extracted key events from Wikipedia¹¹1https://en.wikipedia.org/wiki/Portal:Current_events pages in multiple languages for 2018. Subsequently, we retrieved relevant posts from Twitter or Weibo using event keywords and crawled them to construct the datasets. The events listed on Wikipedia pages in different languages are customized according to users’ preferences in that language, making the obtained dataset closely resemble real-world distributions.

Prior works (Liu et al., 2023; Yang et al., 2023) often partition a single dataset into multiple clients to mimic the federated setting. However, such practices fail to capture the non-IID nature of real-world data. This study breaks this cycle by treating each dataset as an independent client in the experiments. This enables us to replicate the complexities of real-world data distribution across platforms more accurately. By preserving the inherent non-IID characteristics of the data, we aim to enhance the fidelity of our federated learning experiments and provide insights that are more applicable to practical scenarios. In our experiments, we utilize a setup consisting of one server and six clients, each with social message data in distinct languages.

In addition to Local training without parameter sharing, we compare DAMe with two categories of FL methods in the task of SED: (1) Classic FL methods: FedAvg (McMahan et al., 2017) aggregates a weighted global model for all clients. FedProx (Li et al., 2020) introduce regularization to alleviate disparities between the global and local models. (2) Personalized FL methods: Per-FedAvg (Fallah et al., 2020) fine-tunes the global model on the client side to achieve personalization. In Ditto (Li et al., 2021a), each client learns an additional personalized model by incorporating a proximal term to extract information from the updated global model. SFL (Chen et al., 2022) constructs a client graph on the server and aggregates personalized models for each client via structural information. In APPLE (Luo and Wu, 2022), clients have access to all other clients’ models and aggregates locally. FedALA (Zhang et al., 2023) dynamically aggregates the global and local parameters at a fine-grained level based on the local objective.

The experiments are implemented using the PyTorch framework and run on a machine with eight NVIDIA Tesla A100 (40G) GPUs. We randomly sample 70%, 20%, and 10% for training, testing, and validation as common studies on SED (Cao et al., 2021; Ren et al., 2022a). For all methods, we employ the SED model in Section 4.1 as the backbone model. The backbone model consists of two layers of GAT, where each node in the batch aggregates messages from 800 direct neighbors and 100 one-hop neighbors. We set the mini-batch size to 2000, the learning rate to be $1e-3$, the margin for the contrastive triplet loss to 3, and employed the Adam optimizer. During the FL process, we perform 50 communication rounds, and each client conducts local training for 1 epoch, a compromise value across all baselines (Zhang et al., 2023; Chen et al., 2022).

The baseline methods are implemented based on open-source implementations on Github²²2https://github.com/dawenzi098/SFL-Structural-Federated-Learning³³3https://github.com/zs2847037826zs/PFL-Non-IID/tree/master. For SFL, the client-wise relation graph is constructed based on the distances between model parameters using Euclidean distance. It is observed that FedALA failed to converge and entered a state of deadlock after the initial communication round. Therefore, we set a local patience of 10 for FedALA. We set the number of epochs for local training to 50.

For the robustness analysis, we conducted injection attacks involving model poisoning⁴⁴4https://github.com/RingBDStack/KPGNN and data poisoning⁵⁵5https://github.com/thunlp/HiddenKiller following (Zhang et al., 2022) and (Qi et al., 2021), respectively.

All methods utilize k-means clustering. All experiments are repeated 5 times to mitigate the uncertainty of deep learning methods. We report the average value and standard deviation of the 5 repetitions. All implementations are available at https://github.com/XiaoyanWork/DAMe.

Technically, SED involves learning representations of social messages and clustering them into specific events. We evaluate the performance of all methods using three commonly used metrics for clustering tasks: Normalized Mutual Information (NMI) (Estévez et al., 2009), Adjusted Mutual Information (AMI) (Vinh et al., 2009), and Adjusted Rand Index (ARI) (Vinh et al., 2009). These metrics quantify the similarity between the detected and ground truth clusters. A higher score of these metrics indicates better message representation.

This section investigates the performance of SED with various FL systems. The result is demonstrated in Table 2. The results show that DAMe has outperformed all baseline methods on all metrics for each client dataset. Especially on the Arabic dataset, which suffers from limited data samples, DAMe surpasses the baseline methods by at least 0.16, 0.17, and 0.16 regarding NMI, AMI, and ARI, respectively. This observation indicates that DAMe has the potential to integrate most external knowledge from other clients, thereby promoting local performance to the greatest extent. It is observed that compared to local training, DAMe achieves an average gain of 0.07, 0.09, and 0.14 in NMI, AMI, and ARI, respectively, surpassing all FL baseline methods. These results highlight that the proposed framework meets the objective of FedSED, which is to enhance the performance of individual clients and benefit all clients involved. Moreover, in comparison to the pFL methods, the results reveal the following: (1) When comparing methods that directly override the local model with the global model (SFL, PerFedAvg, Ditto), these methods encounter challenges in achieving satisfactory local performance. This is because, in each communication round, training is perceived as starting from scratch on the client side, initialized with the global parameters. In the case of PerFedAvg, fine-tuning on the client side with local data necessitates an additional training round after the FL process. However, during local fine-tuning, there is a risk of catastrophic forgetting, whereby the learned information from the global model can be lost. As for SFL, which aggregates the global model using structural information on the server side, does not significantly contribute to local performance improvement. This is because the objective is to personalize the model with local data, and a model that integrates the majority of external knowledge and overrides the local model disregards the importance of local characteristics, which are crucial factors in personalization. Consequently, SFL does not surpass local training in most cases. (2) When comparing DAMe with methods that locally aggregate models (FedALA and APPLE), it is observed that, for APPLE, exposing each client to other clients’ local models does not necessarily lead to improved local performance. This is because clients cannot accurately determine which models are useful or possess similar distributions to their own. As for FedALA, the performance is relatively satisfactory due to its parameter-level aggregation, which aids in identifying relevant knowledge within the global model. However, FedALA’s approach of dispatching the global model based on weighted averages of data samples does not provide clients with the most advantageous information they require. In contrast, our proposed framework addresses this limitation by considering client similarity during the global aggregation process. This allows the server to dispatch an aggregated global model better aligned with each client’s specific needs. To sum up, the proposed dual aggregation mechanism significantly enhances the performance of local training. This is achieved by considering the local characteristics and providing clients with the knowledge they require the most. The dual aggregation mechanism is crucial in improving local performance to a paramount extent.

To evaluate DAMe’s robustness, we investigate its resistance to two types of attacks: model poisoning attacks and data poisoning attacks. In both attacks, the malicious client possesses the English dataset and aims to compromise the FL process. In the model poisoning attack, the client uploads manipulated parameters to the server (Zhang et al., 2022), resulting in a model with no practical utility. In the data poisoning attack, the client injects backdoors into its own data (Qi et al., 2021) and trains a local model using the poisoned data. Consequently, the injected backdoor affects the model parameters and corrupts the global model. In traditional settings, the server integrates all received models, including the poisoned model, without differentiation, which decreases the accuracy of the global model. Moreover, this corrupted global model is dispatched to individual clients, overriding their local models and compromising their local training. However, the proposed dual mechanism has advantages against such attacks. With BOLA, clients can determine the proportion of the dispatched global model that should be integrated into their local training. This enables them to avoid incorporating potentially poisoned parameters. Additionally, SEGA, implemented on the server side, leverages the client-uploaded models to compute the similarity between clients’ models. This similarity calculation enables the server to develop a dispatching strategy, ensuring that models containing poison parameters are not distributed to clients. From the result in Table 3, it is evident that the performance of the DAMe on all datasets demonstrates a negligible perturbation in performance. This observation indicates that DAMe is resilient to injection attacks in FL. Regarding injection attacks, DAMe has the capability to disregard model injection, as it selectively focuses on relevant information that is essential for clients (while disregarding irrelevant noise).

This section presents an ablation study to identify the contribution of the components in DAMe and analyze the effect of the modules.

The results of the ablation study are presented in Figure 2. The findings demonstrate that all proposed components enhance the performance of FedSED. Specifically, BOLA exhibits a substantial influence on performance improvement. This indicates that in pFL settings, it is crucial to empower clients to determine how much they incorporate knowledge from the global model, while preserving their individual characteristics during training. Moreover, SEGA significantly improves performance and surpasses the baseline SFL, aggregating the global model based on structural information within a client graph. This finding suggests that our approach, which leverages client similarity and minimizes the 2DSE of the client graph for global aggregation, effectively identifies pertinent knowledge that boosts local performance. Lastly, GLECC also plays a role in improving overall performance. This implies that aligning the global and local representations of the same event is advantageous for achieving a consensus between the global and local models, thereby mitigating the inherent heterogeneity.

We analyze the Bayesian search space in the last communication round of the FL process and present the visualization of BOLA in Figure 3 . For all clients, DAMe achieves the best performance. For French, German, and Japanese Twitter, DAMe without BOLA achieves performance above the first quartile of the weight distribution in the search space of the Bayesian optimization algorithm but consistently lags behind the upper bound. This observation highlights the significant benefits of utilizing BOLA to search for local aggregation weights, thereby enhancing local performance.

Here, we delve into the crucial factors in FL, including the convergence, computation and communication overhead of the proposed framework and baseline methods.

Convergence As depicted in Figure 4, the proposed DAMe demonstrates stable convergence. This stability can be attributed to DAMe’s tailored design for the task of SED, which focuses on learning message representations and performing clustering. DAMe stands out as the first work to consider the characteristics of the SED task, unlike baseline methods, which serve as a general framework for FL. This task-specific approach allows DAMe to better suit the requirements of SED, enhancing its performance and convergence compared to the less specialized FL baselines.

Computation Overhead The second column in Table 4 presents the time consumption of all methods during the entire training process. It is observed that DAMe without the GLECC module has the shortest time for training in the FL setting, shorter than DAMe, since it omit the process of obtaining the global and local event representation. Moreover, Per-FedAvg and Ditto exhibit the longest training times, surpassing DAMe’s duration by 1.5 times. This can be attributed to their approaches of local fine-tuning or training additional local models, which significantly prolongs the training process. However, contrasting the results in Table 2, these methods do not demonstrate substantial performance improvements. In this regard, DAMe demonstrates success in such scenarios by achieving notable task performance while maintaining reasonable computational overhead.

Communication Overhead The communication overhead is shown in the last column in Table 4. In most scenarios, the methods adhere to the centralized FL setting, where a single server communicates with all clients, leading to a consistent communication overhead across the same parameter scales. However, APPLE adopts a decentralized FL setting, where all clients interact with each other to determine a local aggregation strategy. This significantly increases the communication overhead as the number of clients grows. In our experiment, where $K=6$, the communication overhead of APPLE is 3.5 times higher compared to other methods.