Toward Practical Entity Alignment Method Design: Insights from New Highly Heterogeneous Knowledge Graph Datasets

Upon conducting an extensive review of the field, we noted that a substantial majority of Entity Alignment (EA) studies, specifically more than 90%, utilize a set of established benchmark datasets for evaluation. These include, but are not limited to, DBP15K datasets, DBP-WIKI, and WIKI-YAGO datasets. These datasets have undoubtedly made substantial contributions to the progression of EA research. However, when applied to the study of HHKGs, the intrinsic limitations of these datasets become increasingly prominent. These limitations largely stem from the inherent characteristics of these datasets. Regrettably, these widely used datasets exhibit a degree of simplification. While these feature may making them convenient for conventional EA studies, it significantly deviates from the high heterogeneity and complexity commonly found in real-world KGs. Consequently, these datasets fail to adequately represent the challenges posed by practical applications. This divergence between the characteristics of benchmark datasets and piratical scenarios poses substantial challenges to the alignment of entities within HHKGs.

In this paper, we delve into the prevalent issues inherent in existing EA datasets by conducting rigorous statistical analyses on widely-used datasets, DBP15K(EN-FR) and DBP-WIKI, proposed by OpenEA (Sun et al., 2020). Our choice to examine DBP15K(EN-FR) is driven by two factors: (1) its exemplary nature as it shares similar characteristics with other datasets, thereby enhancing the generalizability of our analysis; For example, through the statistical analysis (Zhao et al., 2020), we can observed that DBP15K(EN-FR), other DBP15K series dataset, DBP-WIKI, and SRPRS have the same feature that aligned KGs share the similar scale, density, structure distribution, and overlapping ratios. (2) the datasets are extensively employed within the EA research community, making them the representation of benchmark datasets, and its broad acceptance by the EA community ensures the applicability of our findings.

With DBP15K(EN-FR) and DBP-WIKI as our focal point, we assess these datasets from three critical dimensions: scale, structure, and overlapping ratios. Through this systematic exploration, we aim to provide valuable insights into the challenges inherent in existing EA datasets and inform future research directions A comprehensive overview of the dataset statistics is presented in Table 1.

Scale. The statistical analysis of DBP15K(EN-FR) and DBP-WIKI shows that the KGs exhibit similarities in the number of entities, relations, and facts, indicating that the scales of aligned KGs are same. However, in practical scenarios, KGs from different sources are not always same in scale, and even show significant differences. In recent years, there has been research (Sun et al., 2020) focused on aligning large-scale KGs. However, there is still a lack of research and datasets modeling KGs that are significantly different in scale, which widely exist. For example, the ICEWS(05-15) dataset contains more facts (461,329) compared to YAGO (138,056) and WIKIDATA (150,079) dataset (García-Durán et al., 2018). Thus, it is impossible to explore the impact of scale differences between KGs in real-world scenarios.

Structure. The KGs of DBP15K(EN-FR) and DBP-WIKI are similar in density, and also have similar degree distributions as shown in Figure 2, which reflect that the KGs are similar in structure.

To further evaluate the neighborhood similarity between KGs, we propose a new metric called structure similarity, which is the average similarity of aligned neighbors of aligned entity pairs, and details are illustrated in A.3. The structure similarity of KGs in DBP15K(EN-FR) and DBP-WIKI reaches 63.4% and 74.8%, indicating that the entity pairs share similar neighborhoods.

The analysis of the structure of DBP15K(EN-FR) and DBP-WIKI reveals that the structure information of the aligned KGs in previous datasets is similar and easy to leverage, particularly by GNN-based EA methods, such as (Wang et al., 2018; Chen et al., 2022; Mao et al., 2021). These methods exhibit abilities to capture and leverage the structure correlation between KGs, resulting in impressive performances on previous EA datasets. It is worth further investigation to determine if these EA methods, verified on previous datasets, are still effective in piratical scenarios where the structure information of two KGs is significantly different.

Overlapping ratio. We can also notice that the overlapping ratio of DBP15K(EN-FR) and DBP-WIKI is 100$\%$, which refers to another important characteristic of most EA datasets: 1-to-1 assumption (each entity in a KG must have a counterpart in the second KG). As discussed in the recent EA benchmark study (Leone et al., 2022), the 1-to-1 assumption deviates from practical KGs. Especially when the alignment is performed between KGs from different sources (e.g., the global event-related ICEWS and the general KG denoted as WIKI), the entities that can be aligned only make up a small fraction of the aligned KGs, which is manifested by a low overlapping ratio. For example, some non-political entities in YAGO (e.g., football clubs) do not appear in the ICEWS dataset.

In conclusion, through the statistical analysis of DBP15K(EN-FR), we find that the previous EA datasets are oversimplified under the unrealistic assumption (i.e., 1-to-1 assumption) and settings (i.e., the same scale, structure), which are easy to leverage but deviate from real-world scenarios. This one-sided behavior hinders our understanding of the real progress achieved by previous EA methods, especially GNN-based methods, and causes the potential limitations of these methods for applications. Moreover, the heterogeneities between real-world KGs are not only language but also the scales, structure, coverage of knowledge, and others, overcoming the heterogeneities and aligning KGs helps complete and enrich them. Therefore, new EA datasets, which can mimic KGs’ heterogeneities in more practical scenarios, are urgently needed for EA research.

To address the limitations of previous EA datasets, in this paper, we present two new EA datasets called ICEWS-WIKI and ICEWS-YAGO, which integrate the event knowledge graph derived from the Integrated Crisis Early Warning System (ICEWS) and general KGs (i.e., WIKIDATA, YAGO). There is a considerable demand to align them in real-world scenarios. ICEWS is a representative domain-specific KG, which contains political events with time annotations that embody the temporal interactions between politically related entities. WIKIDATA and YAGO are two common KGs with extensive general knowledge that can provide background information. Aligning them can provide a more comprehensive view to understand events and serve temporal knowledge reasoning tasks. The detailed construction processes are illustrated in A.4

Dataset Analysis. An ideal comparison should include KGs as they are. From the EA research view, the proposed datasets sweep the oversimplified settings and assumptions (i.e., KGs always satisfy the 1-to-1 assumption, and KGs are similar in scale and structure) of previous datasets, and thus are closer to the real-world scenarios.

In scale, the original two KGs differ significantly in scale. Correspondingly, the datasets maintained the original distribution of the two KGs during sampling, which means that the constructed dataset maintained the scale difference of the KGs.

In structure, we preserved the feature of the original KGs that the density is significantly different. Figure 2 also shows that the two proposed datasets exhibit highly different degree distributions. The structure similarities are very low (15.4%, 14.0% of ICEWS-WIKI/ YAGO) compared with DBP15K(EN-FR) (66.4%), referring that the KGs of the new datasets are highly heterogeneous in structure.

In overlapping ratio, the datasets mimic a more common scenario that the KGs come from various sources (i.e., domain-specific KGs and general KGs). The differences in the sources are typically not language but the coverage of knowledge, which result in a small proportion of overlapping entities. The EA task on the new dataset is challenging but also reaps huge fruits once realized. As shown in Table 1, the overlapping ratio of the new dataset is exceedingly low compared with DBP15K(EN-FR), which means that the new datasets do not follow the 1-to-1 assumption.

Moreover, an increasing number of KGs like YAGO and WIKIDATA contain temporal knowledge, and two recent studies (Xu et al., 2021, 2022) shows that KGs’ temporal information is helpful.

Through the statistical analysis, we deem that the proposed datasets are highly heterogeneous, which can mimic the considerable practical EA scenarios, and help us better understand the demands and challenges of real-world EA applications. We expect the proposed datasets to help design better EA models that can deal with more challenging problem instances, and offer a better direction for the EA research community.

The proposed datasets are not only meaningful for EA research but also valuable in the field of KG applications. From the KG application view, ICEWS, WIKIDATA, and YAGO are widely adopted in various KG tasks such as temporal query (Xu et al., 2019, 2020a, 2020b; Lin et al., 2022), KG completion (Lacroix et al., 2018; Nayyeri et al., 2021) and question answering (Mavromatis et al., 2022; Saxena et al., 2021). Properly aligning and leveraging these KGs will provide more comprehensive insights for understanding temporal knowledge and benefit downstream tasks.

Translation-based EA Methods. Translation-based EA approaches (Chen et al., 2017; Sun et al., 2017, 2018; Zhu et al., 2017) are inspired by the TransE mechanism (Bordes et al., 2013), focusing on knowledge triplets $(u,r,v)$. Their scoring functions assess the triplet’s validity to optimize knowledge representation. MTransE (Chen et al., 2017) uses a translation mechanism for entity and relation embedding, leveraging pre-aligned entities for vector space transformation. Both AlignE (Sun et al., 2018) and BootEA (Sun et al., 2018) adjust aligned entities within triples to harmonize KG embeddings, with BootEA incorporating bootstrapping to address limited training data.

GNN-based EA Methods. In recent years, GNNs have gained popularity in EA tasks for their notable capabilities in modeling both structure and semantic information in KGs (Wang et al., 2018; Cao et al., 2019; Zhu et al., 2021; Mao et al., 2020b, a). GCN-Align (Wang et al., 2018) exemplifies GNN-based EA methods, utilizing GCN for unified semantic space entity embedding. Enhancements like RDGCN (Chen et al., 2022) and Dual-AMN (Mao et al., 2021) introduced features to optimize neighbor similarity modeling. Recent methodologies, e.g., TEA-GNN (Xu et al., 2021), TREA (Xu et al., 2022), and STEA (Cai et al., 2022), have integrated temporal data, underscoring its significance in EA tasks.

Essentially, GNNs can be considered as a framework incorporating four key mechanisms: information messaging (or propagation), attention, aggregation, and self-loop. Each of these plays a fundamental role in GNNs’ effective performance in EA tasks. We first give a formulation to the general attention-based GNNs as follows:

where $H$ denotes the learned entity embedding, MSG, ATT, AGG, and SELF denote message passing, attention aggregation, and self-loop mechanism, respectively.

Specifically, Message passing (MSG) enables nodes to exchange information and assimilate features from neighbors, encapsulating local context crucial for EA tasks, as illustrated in Figure 3. Attention (ATT) assigns varied weights to nodes during messaging and aggregation, allowing GNNs to focus on more pertinent nodes, thereby enhancing the extraction of essential structural and semantic KG traits. Aggregation (AGG) offers a summarized view of a node’s local context by compiling neighbor information into a concise representation. Lastly, the Self-loop (SELF) mechanism ensures the preservation of intrinsic node features amidst the learning process, even in the face of intricate intra-graph interactions.

Other EA Methods. Certain EA approaches transcend the above bounds. PARIS (Suchanek et al., 2011) is a representative non-neural EA method, which aligns entities in KGs using iterative probabilistic techniques, considering entity names and relations. Fualign (Wang et al., 2023) unifies two KGs based on trained entity pairs, learning a collective representation. BERT-INT (Tang et al., 2020) addresses KG heterogeneity by solely leveraging side information, outperforming existing datasets. However, its exploration of KG heterogeneity remains limited to conventional datasets, not capturing the nuances of practical HHKGs.

In the context of EA in HHKGs, the efficacy of these methods and mechanisms may be influenced by various factors, such as the level of heterogeneity, the number of overlapping entities, and the quality and quantity of structure and semantic information. This highlights the need for an in-depth analysis in HHKG settings, which will be the focus of our subsequent experimental investigation. This investigation is expected to shed light on the real progress made by EA methods and guide future research toward more effective and robust solutions for real-world EA challenges.

To answer Q1, we conducted a detailed analysis in four dimensions (datasets, training strategies, embedding modules, and input features). The main experiment results are shown in Table 2.

From the perspective of datasets, baselines that generally perform well on DBP15K(EN-FR) and DBP-WIKI decrease significantly on new datasets, especially GNN-based and translation-based methods. In practice, existing EA methods are difficult to reach the required accuracy performance for applications on HHKGs.

From the perspective of training strategies, we found that the semi-supervised methods (i.e., BootEA, Dual-AMN(semi), and STEA) did not improve performance compared to their basic models (i.e., AlignE and Dual-AMN(basic)), indicating that the existing semi-supervised strategies are not suitable for HHKGs.

From the perspective of embedding modules, neither translation-based methods nor GNN-based methods struggle to capture the correlation between HHKGs, thus the performance of these models is disappointing on new datasets. Especially, the performances of GNN-based methods, which demonstrate the superiority on the DBP15K(EN-FR) dataset, drop sharply on HHKGs. For example, the SOTA GNN-based model Dual-AMN(basic) (Mao et al., 2021), which performs well on the DBP15K datasets, but only achieved MRR of 0.143 and 0.069 on ICEWS-WIKI and ICEWS-YAGO respectively. The huge performance drops of GNN-based methods responses to Q2. In previous EA datasets, KGs have similar structure information and thus GNN-based EA methods can easily exploit the structure similarities between entity pairs for alignment purposes. By contrast, a pair of identical entities often have diverse neighborhoods in two HHKGs, resulting in poor performances. BERT and BERT-INT, which mainly leverages side information instead of aggregating neighbors, perform better among others (attains MRR of 0.607 and 0.793). FuAlign, which adopt the translation mechanism, performs poorly on HHKGs. Despite not utilizing neural mechanisms, PARIS performed competitively on HHKGs.

From the perspective of input features, the models are capable of utilizing entity name information except Dual-AMN(name) outperform the others, indicating to some extent the importance of entity name information in the context of EA models. Specifically, BERT achieves decent results (MRRs of 0.596 and 0.784) on ICEWS-WIKI and ICEWS-YAGO datasets by directly inputting name embeddings generated by BERT into CSLS similarity (Lample et al., 2018). However, RDGCN and Dual-AMN(name), which also utilize entity name information, perform badly on HHKG datasets. BERT-INT and FuAlign use entity name information without a GNN framework, notably, they perform much better than existing GNN-based EA models, proving that the aggregation mechanism limits their performance on our HHKG datasets. The utilization of structure information on HHKGs does not bring promising performance gains. Whether structure information is essential for EA on HHKGs still remains to be explored. In terms of temporal information, TREA outperforms Dual-AMN with temporal information utilization, proving that temporal information is also valuable in EA tasks when available.

Generally, the efficacy of existing methods is constrained, especially when applied to HHKGs. Specifically, entity name and temporal information play critical roles in alignment, offering a necessary foundation for entity matching across varied KGs. However, under high heterogeneity, the effectiveness of structure information decreases, presenting challenges for traditional EA methods.

To answer Q2 and delve deeper into the performance influence of key components of GNNs (i.e., MSG, ATT, AGG, SELF) mentioned in Section 2.3, we further take the SOTA GNN-based method: Dual-AMN as an example, and devised case studies and ablation studies.

Messaging passing (MSG). To explore the influence of MSG, we conduct case studies as shown in Figure 4.

The experiment reveal that as the structure similarity between entities decreases and their neighbors with alignment labels becomes different, the GNN-based Dual-AMN(name) model can hardly leverage structure information through message passing for EA. From the perspective of label propagation, the performance of GNNs deteriorates because they struggle to propagate the correct alignment labels from pre-aligned entity pairs to unobserved entities.

We also conduct ablation studies to verify the effectiveness of ATT, AGG, and SELF on HHKGs, as shown in Table 3.

Attention (ATT). It plays a critical role in both scenarios, emphasizing the significance of certain nodes and edges in the graph, thus contributing to more accurate alignment decisions.

Aggregation (AGG). The experiment results underscore a dual role of the AGG, which manifests differently depending on the availability of entity name information for EA. In scenarios where entity name information is absent, the elimination of the aggregation mechanism causes a considerable decline in performance. Conversely, when name information is available, the removal of the aggregation mechanism surprisingly improves the model’s performance. This improvement could be attributed to the aggregation mechanism’s propensity to introduce noise within HHKGs, which can obscure the valuable cues embedded in the entity name.

Self-loop (SELF). The self-loop mechanism’s marginal influence suggests its role might be overshadowed by other components, especially in a highly heterogeneous context. However, after removing AGG, the self-loop mechanism serves a critical function by maintaining each node’s individual features, acting as a form of identity preservation and feature selection in the face of HHKGs.

Through this analysis, it becomes evident that major factor causing a drastic decrease in performance is the challenge faced by the message passing and aggregation mechanisms of GNNs on HHKGs. These mechanisms struggle to propagate and aggregate valuable information amidst the influx of unrelated data. This revelation highlights the need for more in-depth analysis and more nuanced, data-aware approaches in model design and applications, especially when dealing with highly heterogeneous settings.

Entity Name Encoder. We first adopted BERT (Devlin et al., 2019) to encode the entity names into initial embeddings. Then we introduced a feature whitening transformation proposed by (Su et al., 2021) to reduce the dimension of the initial embeddings. The integration of BERT and the whitening layer allows the model to effectively capture the semantic meaning of entities, without requiring additional supervision. Finally, we adopted a learnable linear transformation $W_{\mathcal{T}}$ to get the transformed entity name embeddings $\{{\textbf{h}^{name}_{n}}\}^{N}_{n=1}$.

Entity Time Encoder is designed to verify the power of temporal information, building upon evidence of its effectiveness as demonstrated in the comparative experiments of Dual-AMN and TREA. Simple-HHEA first annotates the time occurrence of each entity according to facts in KGs. Specifically, for our proposed HHKG datasets, the time set $T$ ranges from 1995 to 2021 (in months) of KGs. The encoder can obtain a binary temporal vector for entity $e_{n}$ denoted as $\mathbf{t}_{n}=\left\{\mathbf{t}_{n}^{i}\right\}_{i=1}^{|T|}$, where $\mathbf{t}_{n}^{i}=1$ if facts involving $e_{n}$ happen at the $i^{th}$ time point, otherwise $\mathbf{t}_{n}^{i}=0$. We adopted Time2Vec (Goel et al., 2020) to obtain the learnable time representation $\textbf{t2v}(t)[i]$, expressed as:

where $\textbf{t2v}(t)[i]$ is the $i^{th}$ element of $\textbf{t2v}(t)$, here we adopt $\cos(\cdot)$ as the activation function to capture the continuity and periodicity of time, $\omega_{i}$s and $\varphi_{i}$s are learnable parameters.

Then, the encoder sums the time representations obtained by Time2Vec of the entity occurrence time points, and gets entity time embeddings $h^{time}$ through a learnable linear transformation $W_{\mathcal{T}}$. Unlike other methods that strictly correspond to time, Time2Vec can model the cyclical patterns inherent in the temporal data.

Simple-HHEA⁺. To study the use of the structure information in EA between HHKGs, based on basic Simple-HHEA, we introduced an entity structure encoder, which is different from the aggregation mechanism for modeling structure information. To synchronously model the one-hop and multi-hop relations, the encoder employs a biased random walk with a balance between BFS and DFS (Wang et al., 2023). Denote $e_{n}$ as the selected node at $i$ th step of random walks, and $\text{PATH}_{n}=$ $\left(e_{1},r_{1},e_{2},\ldots,e_{i-1},r_{i-1},e_{i}\right)$ as the generated path. The probability of an entity being selected is defined as:

where ${\mathcal{N}_{e_{i}}}^{-}$ denotes entity $e_{i}$ ’s 1-hop neighbors $N_{e_{i}}$ except $e_{i-1}$; $d\left(e_{i-1},e_{i+1}\right)$ is the length of the shortest path between $e_{i-1}$ and $e_{i+1}$; $\beta\in(0,1)$ is a hyper-parameter to make a trade-off between BFS and DFS (Wang et al., 2023). Once an entity $e_{i+1}$ is selected, the relation $r_{i}$ in the triple $\left(e_{i},r_{i},e_{i+1}\right)\in T^{\prime}$ is selected simultaneously.

Then, the Skip-gram model $SkipGram(\cdot)$ together with a linear transformation $W_{\mathcal{D}}$ are employed to learn entity embeddings $\{{{dw}_{n}}\}^{N}_{n=1}$ based on the generated random walk paths to capture structure information of KGs. It eliminates the aggregation or message-passing mechanisms and doesn’t require supervision.

The linear transformations adopted in each encoder bear similarities to the self-loop mechanism in GNNs, a mechanism that has been validated for its effectiveness in previous experiments. By employing linear transformations, the Simple-HHEA model aims to be adapted to different data situations. Finally, the multi-view embeddings of entities are calculated by concatenating different kinds of embeddings, expressed as:

where $\otimes$ denoted the concatenation operation.

We adopt Margin Ranking Loss as the loss function for training, and Cross-domain Similarity Local Scaling (CSLS) (Lample et al., 2018) as the distance metric to measure similarities between entity embeddings.

This section aims to uncover the essential EA model needed in practical scenarios through a comprehensive comparison of Simple-HHEA, Simple-HHEA⁺, and other baseline methods.

Simple-HHEA vs. baseline methods. As a critical tool for the in-depth analysis in this section, we will first examine the performance of Simple-HHEA. Table 2 shows the comparison results. Compared with the current SOTA methods: BERT-INT and PARIS, Simple-HHEA and Simple-HHEA⁺ achieve competitive performances on the previous datasets: DBP15K(EN-DE) and DBP-WIKI. For the two HHKG datasets, we can observe that Simple-HHEA and Simple-HHEA$+$ outperform baselines (including BERT-INT) by $15.9\%$ Hits@1 on ICEWS-WIKI, and $9.1\%$ Hits@1 on ICEWS-YAGO.

The introduction of Simple-HHEA aims to guide the design of future EA models. While it is straightforward in design, experimental comparisons affirm that a well-conceived simple model can also achieve commendable results. We also conduct an efficiency analysis as shown in Table 4. It corroborates the efficiency superiority of Simple-HHEA over baselines, which indicates that enhancing model efficiency through a simpler design is also essential in practical, rather than pursuing accuracy without considering complexity.

Ablation Studies. We performed ablation studies by ablating Simple-HHEA to analyze how and when each component of it adds benefit to the EA task as shown in Table 5. It is worth noting that the whitening strategy contributes significantly (5.8% and 6.1% improvement on Hits@1) to the great performance of our model, and the temporal information also brings the performance improvement (1.9% and 1.8% on Hits@1). By contrast, the introduction of structure information of Simple-HHEA⁺ degrades the performance.

Questioning whether entity name information always holds value, and if structure information is indeed ineffective on HHKGs, we conducted sensitivity experiments.

The impact of structure information.  To ponder how structure information affects EA on HHKGs, we selected several SOTA GNN-based methods (which mainly leverage structure information) and Simple-HHEA, and then randomly masked proportions (0% ~80%) of facts in KGs to mimic different graph structure conditions.

As shown in Figure 5, the mask of structure information affects the overall performance of GNN-based EA methods on both datasets with the mask ratio of facts increasing. This phenomenon proves that GNN-based EA methods’ performance mainly depends on the structure information, even on HHKGs. The structure information is difficult to exploit in HHKGs due to their structure dissimilarity, while they still learn a part of meaningful patterns of the HHKGs’ structural information for EA, which indicates that existing GNN-based EA methods still have much room for improvement by better leveraging structure information in HHKGs.

For Simple-HHEA, with the mask ratio of graph structure gradually increasing, the temporal information is lost, resulting in the performance decreases of Simple-HHEA. It is worth noting that as the mask ratio of structure information increases, the performance of Simple-HHEA⁺ improves. At a mask ratio of 60%, Simple-HHEA⁺ even outperforms the basic version. This reflects that in HHKGs, the overly complex and highly heterogeneous structure information will introduce additional noise, and distract the structure correlation mining of Simple-HHEA⁺ for EA.

In conclusion, the structure information should not be ignored, especially when the quality of other types of information cannot be guaranteed. Future EA methods should consider strategies for pruning and extracting valuable cues from highly heterogeneous structural data for effective EA.

The impact of entity name information. The phenomenon observed in Table 2 is that both GNN-based methods (e.g., Dual-AMN(name)) and other methods (e.g., BERT-INT, FuAlign, Simple-HHEA) can utilize same entity name information, but their performance varies widely. To explore the role of the entity name information for EA, we randomly mask a proportion (0% ~100%) of entity names, for mimicking different entity name conditions.

As shown in Figure 6, while the mask ratio of entity names gradually increases, the performances of the EA methods which highly rely on entity name information, drop sharply. For BERT-INT, although it uses neighbor information in its mechanism setting, it still needs to filter candidate entities through entity name similarities, so the performance decreases vividly. In addition to entity name information, FuAlign learns structure information through TransE (Bordes et al., 2013). Therefore, when entity name information is gradually masked, its drop is much smaller than BERT-INT, which indicates that the use of structure information plays a more prominent role in FuAlign, and slows down the performance degradation. Dual-AMN(name)’s performance does not change significantly when the entity name information is masked, which shows that Dual-AMN(name) can leverage structure information to achieve a stable performance when entity name information is absent.

As the mask ratio of entity name gradually increases, the basic Simple-HHEA also drops sharply like other baselines which highly rely on the quantity of entity name information. Notably, as the quality of entity name information declines, the role of the entity structure encoder in Simple-HHEA⁺ becomes prominent, even surpassing the basic version when the mask ratio is $80\%$. It indicates that the structure information should not be ignored in EA, especially when the quality of other information can not be guaranteed.

We further designed experiments to explore the performance of Simple-HHEA⁺ when both name and structure information are changed. As shown in Figure 7, when the quality of entity name information is high (the name mask ratio is less than 80%), the performance of the model decreases when reducing the mask ratio of the structure information; Notably, when the quality of entity name information becomes very low (mask ratio reaches 80%), the above phenomenon is reversed, and the performance increases when introducing entity structure information. This phenomenon reflects that when the quality of different types of information changes, our proposed Simple-HHEA⁺ can also achieve self-adaptation through simple learnable linear transformations.

In summary, the design of effective EA methods requires the ability to exploit various types of information and adapt to different levels of information quality.

It’s crucial to develop more datasets that closely resemble real-world entity alignment scenarios. As analyzed in Section 2.1, designing datasets that model different practical conditions is essential for researching practical EA methods. The two datasets proposed in this paper focus on the highly heterogeneous nature of KGs in EA. The reflections on EA method design based on these datasets lead us to conclude that datasets are foundational to EA research before designing new methods.

Attention could be given to the design of message propagation and aggregation mechanisms in GNNs or to proposing new EA paradigms different from existing GNN approaches. Detailed analysis in Sections 3.2 and 4.2 reveals limitations in existing message propagation and aggregation mechanisms, especially in highly heterogeneous KG scenarios. Two directions can be considered: designing new GNN methods with a focus on MSG, ATT, AGG, and SELF mechanisms, or proposing entirely new paradigms different from existing GNN schemes.

(1) For new GNN method design, experiments in Sections 3.2 and 4.2 suggest insights for future GNN-based EA method design. Based on Figure 4, we find that existing GNNs struggle with accurate alignment in highly heterogeneous structural scenarios. Additionally, as shown in Figure 5, reducing structural information surprisingly improves the performance of Simple-HHEA, even surpassing versions that don’t utilize structural information. This counterintuitive phenomenon can guide us in designing new GNN methods’ message propagation and aggregation modules. For instance, models could automatically prune neighbors or combine relational and temporal information to design new attention mechanisms that identify valuable neighbors for EA, reducing the negative impact of aggregating noisy information. Furthermore, analysis of the Self-loop mechanism in Section 3.2 highlights its value in preserving node-specific information and feature selection. Optimizing this mechanism in new GNN designs could enhance its feature selection capabilities and improve EA effectiveness.

(2) For proposing new paradigms different from existing GNN schemes, our experiments reveal that various non-GNN models, even traditional probabilistic models like PARIS, achieve good results. Thus, EA method design need not be confined to GNN architectures. Notably, recent studies like BERT-INT and Fualign have also acknowledged this, though they still have limitations. This direction offers rich research potential, such as combining neural-symbolic methods (noting PARIS’s competitive performance in our experiments) and integrating knowledge graphs with large language models.

Effective utilization of name information can mitigate the negative impacts of high heterogeneity in graphs, but the use of structural and temporal information, especially when the quality of name information is uncertain, could not be overlooked. EA methods need to adapt to varying feature quality conditions. In highly heterogeneous knowledge graphs, effective utilization of name information significantly improves EA performance. However, combining separate and comprehensive analyses of each feature type in Section 4.2 of EA on HHKGs reveals the importance of utilizing structural and temporal information. Simultaneously, designing adaptive feature fusion modules to face different feature conditions and maintain stable performance is essential.