Text Classification using Graph Convolutional Networks: A Comprehensive Survey

In general, text classification can be applied at multiple levels of document hierarchy as shown in Fig 1.

• •

  Word level: This is the most common level of text classification, where the goal is to classify individual words into categories such as topics, sentiment, or named entities.

• •

  Phrase level: This involves classifying groups of words that function as a unit, such as noun phrases or verb phrases. This level of granularity is usually required in applications pertaining to sentiment classification where it would not be correct to assume that the sentiment reflected in an entire document or paragraph would remain consistent (Litman, 1996; Tan et al., 2012). A frequent challenge while attempting classification at this level is the need for extensive manual annotation for each phrase and therefore, researchers have resorted to semi-supervised methods (Nam et al., 2009).

• •

  Sentence level: At a higher level of abstraction, sentence classification seeks to predict labels for each complete sentence (Guo et al., 2019; Hassan and Mahmood, 2018; Wieting and Kiela, 2019). Similar to phrase-level classification, this also enables us to derive a more meaningful description of a document by inferring distinct ideas and arguments contained within sentences instead of considering the entire text in broad strokes. As an example, this can be useful in identifying not just different speakers in a written dialogue but also changes in their attitudes and emotions.

• •

  Paragraph level: Paragraphs or subsections of a document are classified in to a set of categories. A paragraph can have one or more sentences and can even span an entire document. Learning distributed vector representations for pieces of texts of variable length is an important research direction (Le and Mikolov, 2014).

• •

  Document level: A significant bulk of text classification has been dedicated to document classification. At this level, the text classification algorithm predicts the classes of the entire document, i.e., the topic of an article or the genre of a book. In recent times, neural network-based approaches (Akhter et al., 2020; Yao et al., 2019) have dominated this domain. Researchers have also attempted to exploit BERT (Adhikari et al., 2019) for this purpose despite its prohibitively large number of parameters and computational overhead.

Text classification has broad applications in Natural Language Processing (NLP). The most frequent examples include:

• •

  Sentiment Analysis: Sentiment analysis classifies text as positive, negative, or neutral. It’s widely used in social media monitoring, customer feedback analysis, and product review categorization. Methods like Decision Trees, Naive Bayes, and Maximum Entropy have shown high performance (Hemalatha et al., 2013). Moreover, deep learning techniques using CNNs (Liao et al., 2017), RNNs (Can et al., 2018), and their combinations (Basiri et al., 2021; Wang et al., 2016a) have also achieved promising results.

• •

  Content Moderation: Automated text classification is used to detect and flag inappropriate content on social media and online communities, including hate speech (Gambäck and Sikdar, 2017; Del Vigna12 et al., 2017), offensive language (Hajibabaee et al., 2022), adult content (Kim and Nam, 2006), and privacy protection (Liang et al., 2024), ensuring a safer user experience.

• •

  Spam Filtering: Text classification helps prevent unwanted communications from reaching a user’s inbox by classifying messages as spam or not. Naive Bayes, along with SVM (Sculley and Wachman, 2007), Decision Trees (Sharma and Sahni, 2011), and Random Forests (Sjarif et al., 2019), have proven effective in spam filtering.

• •

  News Categorization: ML automates the categorization of news articles into topics like sports, politics, and entertainment. SVM-based methods have been used for both pre-defined (Krishnalal et al., 2010; Dadgar et al., 2016) and user-defined categories.

Over the years, a wide variety of machine learning techniques, ranging from classic techniques to deep neural networks have been leveraged for the task of text classification. The more traditional approaches include Naïve Bayes (Dai et al., 2007; Chen et al., 2009), Support Vector Machines (SVM) (Sun et al., 2009) and Decision Trees (Bahassine et al., 2016) as well as various ensembles and combinations of these approaches (Onan, 2017). More recent works indicate a notable transition to different neural network architectures such as Recurrent Neural Networks (RNN) (Liu et al., 2016), and Long short-term memory (LSTM) (Liu and Guo, 2019; Zhou et al., 2016). RNNs can assign more weightage to previous data points in a sequence, which makes them suitable for understanding semantics and context while performing text classification. However, they tend to suffer from vanishing and exploding gradients, which makes it difficult for them to preserve long-term dependencies. An LSTM is a special kind of RNN that is not vulnerable to these problems and better suited for such scenarios (Sherstinsky, 2020).

Convolutional Neural Networks (CNN) (O’Shea and Nash, 2015), although originally developed for image processing, have proven to be useful for text classification as well (Wang et al., 2018a; Liu et al., 2017a). In a typical CNN, the input undergoes convolution with learnable feature maps that can be layered to offer multiple filters on the input. To decrease computational load, CNNs use pooling to reduce the output size from one layer to the next in the network. The final layers in a CNN are typically fully connected.

More recently, transformer models have demonstrated great prowess in handling a wide range of NLP tasks. Transformers (Vaswani et al., 2017a) feature an encoder-decoder structure that relies on attention to generate an output. The encoder first maps an input sequence to a sequence of continuous representations. The decoder then takes these along with its own output at the previous time step to generate an output. Some of the more commonly used transformer-based models for text classification include BERT (Sun et al., 2019), XLNet (Chang et al., 2020), and RoBERTa (Briskilal and Subalalitha, 2022).

Bidirectional Encoder Representations from Transformers (BERT), in particular, has achieved amazing results in many NLP tasks and text classification is no exception (Zeng et al., 2024). BERT consists of multiple encoder blocks from the transformer stacked atop one another. Unlike many other transformer models, BERT is able to incorporate context on both sides of a word to gain better results. Since BERT is pre-trained on general tasks, it does not require huge amounts of additional data to fine-tune it for a particular target task. Graph Neural Networks (GNN) operate on graphs and are able to model global information in corpora. There exist several variations of GNNs, namely Graph Convolutional Networks (GCN) (Kipf and Welling, 2016a), Graph Attention Networks (GAT) (Veličković et al., 2017; Pal et al., 2020), Graph Auto-encoders (Kipf and Welling, 2016b), Gated GNNs (Li et al., 2015), and GraphSAGE (Hamilton et al., 2017). Besides these GNN architectures, there are also Graph Transformers (Yun et al., 2019; Zhang and Zhang, 2020) that embed a graph structure into the transformer architecture (Vaswani et al., 2017a), enabling learning from the entire graph rather than just the local neighborhood.

Among the various deep learning models that operate on graph data, GCNs, in particular, have shown great promise with regard to text classification (Yao et al., 2019). One of their key strengths is their ability to factor in the global information between words and concepts by performing convolution operations on neighboring nodes in a graph to aggregate information from a node’s neighbors and update that node’s representation.

While there exist several other surveys on the topic of general machine learning and deep learning text classification techniques (Aggarwal and Zhai, 2012; Ikonomakis et al., 2005; Kowsari et al., 2019; Minaee et al., 2021), as well as those pertaining to graph convolutional networks and graph neural networks (Zhang et al., 2019b; Zhou et al., 2020a; Wu et al., 2020b), this article, to the best of our knowledge, is the first attempt at extensively and exclusively reviewing various state-of-the-art methodologies using GCNs for text classification. Significant strides have been made in this area of research in the last few years and we have therefore placed particular emphasis on the more recent GCN-based text classification methods, their conception and various strengths and weaknesses.

Text classification can be performed using two types of learning mechanisms:

• •

  Inductive Learning: This mode of learning takes place in two phases, namely the training phase and the inference phase. In the training phase, we use a training set to build a machine learning model, while during inference, we generalize this model to a separate, previously unseen test set to predict its labels.

• •

  Transductive Learning: In transductive learning, both training and inference occur simultaneously. In other words, both training and test data are available to us at the same time and we make use of patterns present in both sets and labels of the training set to infer the unknown labels of the test set.

Generally, transductive learning is more computationally expensive as it requires the algorithm to rerun to infer the labels of any new datapoints whereas in inductive learning, we build a generalized predictive model beforehand that does not require retraining for inference.

From our comparative analysis of various GCN-based text classification approaches, several key findings have emerged:

• •

  Initially, the focus was predominantly on supervised learning approaches. Early methods such as TextGCN (Yao et al., 2019) showcased remarkable performance and underscored the potential of GCNs in text classification tasks. These approaches laid the groundwork for subsequent innovations by demonstrating how graph-based representations could capture semantic relationships within text data.

• •

  In terms of architecture, earlier innovations leveraged optimization-centric methods and multigraph approaches to enhance graph representations whereas more recent architectures integrate GCNs with advanced models such as BERT and other LLMs to further enhance text classification performance.

• •

  As the field progressed, there was a noticeable shift towards semi-supervised methods, motivated by the need to leverage large amounts of unlabeled data, which is often more readily available than labeled datasets.

• •

  More recently, the research focus has shifted towards self-supervised learning, reflecting a broader trend in machine learning towards minimizing the dependency on labeled data.

• •

  Overall, the performance on benchmark datasets has improved significantly as models have evolved from basic GCNs to more complex and hybrid architectures over the years.

In this survey, we adopted a rigorous criteria to evaluate and compare different GCN-based approaches for text classification. Specifically, we focused on seminal works as well as recent papers from the last five years that are well-cited and published in reputable venues. This approach allowed us to incorporate both historical context and the latest advancements in the field. Datasets commonly used across multiple studies are included in this review to ensure consistent comparisons and to yield meaningful insights regarding their relative strengths and limitations.

The key contributions of our work are outlined as follows:

• •

  While there are several reviews on GCNs, this survey uniquely provides an exhaustive and categorical breakdown of GCN-based text classification techniques, focusing on the architecture and mode of supervision.

• •

  Based on supervision, we categorize existing methodologies into supervised, semi-supervised, self-supervised and weakly supervised techniques.

• •

  Based on architecture, we categorize these as fundamental techniques and GCN integration with generative models. In the latter we discuss combination of GCN with RNNs, LSTMs, BERT and LLMs, offering a structured understanding of the field.

• •

  Detailed analysis and comparison of various methods based on their performance on popular, benchmark datasets are provided to highlight their strengths and limitations.

• •

  Future research directions and challenges in GCN-based text classification are discussed to guide further advancements in the field.

The remainder of this survey paper is organized as follows. Section 2 provides a detailed review of the existing surveys. Section 3 provides insight into various textual embeddings. Section 4 provides an overview of the GCN architectures for text classification. Section 5 categorizes GCN architectures based on integration with generative models. Section 6 categorizes existing GCN approaches as supervised, semi-supervised, self-supervised, and weakly supervised. Section 7 covers the performance comparison, datasets, metrics and analysis of results. Finally, we conclude this paper in Section 8 and also discuss future research directions.

Let $G=(V,E)$ be a graph where $V$ and $E$ represent sets of nodes and edges respectively. Each node $v_{i}$ in the graph has a corresponding m-dimensional feature vector $x_{i}\epsilon R^{n}$. If we have $n$ nodes, this translates to a feature matrix $X\epsilon R^{nxm}$ that serves as the input to the first GCN layer. In supervised classification, the labels of only a subset of nodes is known and the objective is to predict the remaining unknown labels.

Adjacency matrix $A$, the Degree matrix $D$, and the normalized adjacency matrix $\hat{A}$ are often used as Graph Shift Operators (GSO). $A$ is a sparse matrix with non-zero entities with all the diagonal elements set to one since each node is assumed to be connected to itself. For its remaining elements, we set $A_{ij}=w_{ij}$, for all ($i$, $j$) where $w_{ij}$ is the edge weight between nodes $i$ and $j$. $D$ is a diagonal matrix with degrees as diagonal entries, i.e., $D_{ii}=d_{i}$. The normalized adjacency matrix is given by $\hat{A}=D^{-1/2}AD^{-1/2}$.

A GCN is a multilayer neural network that can be applied directly to graphs, yielding vectors of nodes based on their connections with other nodes and overall graph topography. For a single layer GCN, a graph convolution can be represented as: $H_{1}=\rho(\hat{A}XW_{0})$, where $W_{0}$ is a learnable weight matrix for the first layer and $\rho$ is an activation function like ReLU. Such a GCN can only update a node’s representation based on information captured from its immediate neighbors. To leverage information from distant neighbors, multiple GCN layers can be stacked as given: $H_{k+1}=\rho(\hat{A}H_{k}W_{k}).$

A k-layer GCN as depicted in Figure 3 can thus allow message passing among nodes that are at maximum k hops away, which tends to have a similar effect as an increased receptive field in deeper CNNs. Successive convolutions smoothen the resulting hidden representations locally along the edges of the graph, resulting in similar predictions among closely connected nodes. Typically, 2-layer GCNs have been used in literature as a higher number of layers yields diminishing returns and even worse performance due to over-smoothing. Predictions are made using a softmax classifier at the output whereas cross-entropy error over all labeled nodes is used as the loss function.

Graph-based approaches for text classification have long existed (Angelova and Weikum, 2006). Defferrard et al. (Defferrard et al., 2016) were among the first to extend and demonstrate the efficacy of Spectral Graph Convolutional Neural Networks for this task. They framed their problem as a semi-supervised node classification problem where labels were only available for a small subset of nodes and based it on the assumption that connected nodes likely share the same label. They leveraged a global graph with Word2Vec embeddings and fast K-localized convolutions were applied.

Kipf and Welling (Kipf and Welling, 2016a) put forward a solution based on spectral GCNs featuring fast localized convolutions. They also considered transductive node classification in notably larger networks. They demonstrated that with a layer-wise propagation rule along with fewer parameters and operations, both scalability and classification performance can be improved in large-scale networks. This method proved to be a seminal work in this domain as it formalized GCNs as we know them and laid the foundation for all GCN methods to come. Besides being extensively applied to word-document graphs for text classification in subsequent literature, several works have also built upon this architecture for semi-supervised node classification. Specifically by introducing a discriminative hierarchical convolutional mechanism (Jin et al., 2021b), incorporation of high-order proximities using quantum information theory (Shah et al., 2021), integrating with a neural topic model (Yu et al., 2021), and including more robust privacy measures (Igamberdiev and Habernal, 2021).

Yao et al. (Yao et al., 2019) built upon the work of (Defferrard et al., 2016) and (Kipf and Welling, 2016a) by leveraging a singular heterogeneous text graph for the entire corpus and further extended them through the inclusion of document nodes and the representation of relationships among word and document nodes using suitable metrics such as Pointwise mutual information for word-to-word relations and TF-IDF for word-to-document relations as shown in Fig. 4. They then modeled this graph with GCN (Kipf and Welling, 2016a) so it may automatically learn embeddings of both words and documents in conjunction, supervised by a small subset of labeled documents. The text graph construction method and application of a two-layer GCN presented in this paper has influenced many later works (Huang et al., 2019; Wu et al., 2019; Zhu and Koniusz, 2020; Lei et al., 2021; Wang et al., 2023a; Liu et al., 2020; Zhou et al., 2020b; Wang et al., 2021b; Dai et al., 2022; Wang et al., 2023b; Zhang et al., 2020; Wang et al., 2022a; Cai et al., 2020; Ma et al., 2021; Liu, 2022; Ye et al., 2020; Yang et al., 2021b; Zhao et al., 2022; Cui et al., 2022; Li et al., 2021; Liu et al., 2021; Ragesh et al., 2021; Wang et al., 2022b; Marreddy et al., 2022; Zhu et al., 2021a; Yang et al., 2022; Wu et al., 2023) as discussed in the following sections.

Some researchers have sought to use Convolutional Neural Networks (CNNs) with GCN, owing to the former’s ability to capture local contextual information effectively. Zeng et al. (Zeng et al., 2022) proposed a GCN-CNN boosting ensemble in which a CNN learned from examples misclassified by a GCN to improve performance. Similarly, in addition to presenting a hybrid GCN-Bi-LSTM model, Yang et al. (Yang et al., 2021a) also introduced variant models (ServeNet, C-LSTM) that used 1-D or 2-D CNN layers in addition to the Bi-LSTM layers to better capture local information.

GCNs have been used in tandem with recurrent neural networks (RNN) and their variants, such as Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU). The primary goal here is to enable the overall architecture to effectively capture long-range and short-range contextual dependencies. Tang et al. (Tang et al., 2020a) combined textual and part-of-speech features obtained using Bi-LSTM with an adjacency matrix capturing the dependency relationship to address the problems of contextual dependency and lexical polysemy in GCNs. Gao et al. (Gao et al., 2020) also proposed a GCN and LSTM hybrid structure that combined outputs of each GCN layer with an embedding generated by passing them through an LSTM. Meanwhile, Yang et al. (Yang et al., 2021a) leveraged a weighted combination of evaluation values from GCN and Bi-LSTM classifiers to improve text classification performance. Similarly, Zhu et al. (Zhu et al., 2021b) utilized a Bi-LSTM to capture the local structure information of a sentence and a text graph and GCN to model the global dependency information between words. Both global and local dependency structure signals were then fused using an attention mechanism and used to guide the training process.

In terms of performance, GRUs are generally faster and less complex than LSTMs because they have fewer gates. Thus, Dong et al. (Dong et al., 2022) opted to use a two-way GRU model in tandem with a GCN in their proposed architecture. They fed Word2Vec embeddings into a BiGRU layer, yielding a representation that captured the global contextual features and long-range dependencies within the text. This representation was then passed through a GCN layer to extract complex semantic relations and spatial feature information. The GCN output was then fed into a classifier that predicted the class label of the input text.

Liu et al. (Liu et al., 2023) proposed a method that combined Transformer with GCN for improved semantic representation of documents. After the text graph underwent the first graph convolutional layer, its word nodes were fed into a transformer to capture the contextual and sequential information of the text. The graph was then augmented with the transformer’s outputs and passed through a second graph convolutional layer to ultimately yield the final classification results. TLC-XML (Zhao et al., 2024) is a Transformer-based model for extreme multi-label classification. It employs GCNs for cluster correlation learning.

Lu et al. (Lu et al., 2020) augmented GCN’s capacity to model global information about the vocabulary of a language with BERT’s ability to capture the local contextual information within a sentence or document and proposed a solution that outperformed either of its individual components as evident by their experiments on various state-of-the-art text classification datasets. They first built a GCN on the vocabulary graph based on word co-occurrence information, similar to approaches in (Defferrard et al., 2016; Kipf and Welling, 2016a; Wu et al., 2019), and then passed the word embedding and the relevant part of the graph embedding together to a self-attention encoder in BERT. This enabled them to interact and guide each other while learning the classifier such that the resulting representation could harness local and global information when performing text classification (see Fig. 6). Xue et al. (Xue et al., 2021) applied a GCN to a text graph constructed based on NPMI and WordNet to obtain a hidden state representation that could effectively capture global contextual dependencies and semantic information. An attention mechanism was then used to combine this with the local information extracted using BERT. Alternatively, Gao and Huang (Gao and Huang, 2021) and She et al. (She et al., 2022) used a gating mechanism to integrate BERT and GCN embeddings. Chen et al. (Chen et al., 2022) proposed a relational graph convolutional network to process the semantic features obtained by BERT representation as part of a text-based accident causal classification method. The R-GCN captured immediate syntactic neighbor information of each word and assigned different weights to different types of edges. They also introduced a gate mechanism to reduce the influence of false dependency edges caused by the domain gap. Lin et al. (Lin et al., 2021) proposed a method that first constructed a heterogeneous graph for the corpus with nodes representing either words or documents, similar to TextGCN. However, the document node embeddings were initialized with pre-trained BERT representations. Thus, by jointly training the BERT and GCN modules, this model could take advantage of both BERT’s large-scale pretraining and GCN’s message propagation mechanism, the combined effectiveness of which had not been previously explored. The resulting model was able to obtain state-of-the-art performance on a wide range of text classification datasets. More recently, Li et al. (Li et al., 2023) used a pre-trained BERT model to initialize document nodes in their heterogeneous graph structure that also included word and entity nodes. They generated entity nodes by mapping the entities in the text to an exogenous knowledge base. While word-word and document-word edges were modeled as in (Yao et al., 2019), an attention mechanism was used to weigh document-entity edges and personalized PageRank to measure the semantic relatedness of entity-entity edges. An enhanced GCN with graph sampling and DropEdge techniques was then applied to mitigate the problems of neighbor explosion and noisy inputs in GCN, and the final node embeddings were used for text classification.

Several hybrid GCN architectures have also been explored in recent research. For instance, various authors have leveraged GCN, BERT, and Bi-LSTM simultaneously to improve text classification performance. Zeyu et al. (Zeyu et al., 2021) used BERT to obtain word representations from long texts and fed them into a Bi-LSTM model to capture their semantic relationship. A GCN was then applied to a graph constructed using these word features as nodes and vectors similarity between them as edges. Xue et al. (Xue et al., 2022) also used BERT embeddings as input features along with one graph based on NPMI and WordNet and the other based on dependency relationships. GCNs were trained on these graphs separately. The resulting hidden states were concatenated, fused with the input BERT features, and fed into a fusion model composed of two Bi-LSTM layers with an attention layer in between to generate the final features for classification.

The synergy between GCNs and the LLMs in NLP enhances performance in text classification. Zhou et al. (Zhou et al., 2024) proposed Clip-GCN multimodal fake news detection, which leveraged the CLIP (Radford et al., 2021) pre-training model to extract joint semantic features from image-text information. The model utilized adversarial neural network to extract inter-domain invariant features and employed GCNs to capture intra-domain knowledge for detecting emergent news. Chen et al. (Chen et al., 2024) used, gpt-3.5-turbo-0613 (OpenAI, 2023) as LLM, in graph machine learning for text classification. They investigated two approaches: LLMs-as-Enhancers and LLMs-as-Predictors. The former enhanced nodes’ text attributes with LLMs’ knowledge and generated predictions using MLP, GCN, and GAT, comparing their performance. Li et al. (Li et al., 2024) proposed a hybrid approach that combined Natural Language Inference (NLI) and Graph Convolutional Networks (GCNs) for ontology completion. The GCN models used ConCN concept embeddings as input features and achieved strong performance, with the GCN and Llama2-13B (Touvron et al., 2023a) variant. Du et al. (Du et al., 2024) proposed a Graph-aware Convolutional LLM method aimed at enabling LLMs to capture high-order relations within user-item graphs using textual data. This method employed the LLM as an aggregator in graph processing to facilitate a step-by-step understanding of graph-based information. Specifically, the approach leveraged ChatGPT to enhance descriptions by systematically exploring multi-hop neighbors layer by layer, thus progressively propagating information throughout the GCN.

The integration of GCNs with generative models has emerged as a powerful approach for enhancing text classification performance. Various hybrid architectures are developed, leveraging the strengths of GCNs in modeling relational structures and the advanced contextual understanding provided by models like BERT, LLMs, and LSTMs. This integration have consistently led to improvements in accuracy and robustness across diverse datasets. Early models combining GCNs with BERT, such as VGCN-BERT (Lu et al., 2020) and GC-GCN-BERT (Gao and Huang, 2021), successfully merged global and local text features, resulting in superior performance. Subsequent models, such as R-GCN (Chen et al., 2022) and BERT-GCN (Lin et al., 2021), further enhanced this synergy by introducing gating and attention mechanisms. The integration of GCNs with LLMs represents a more recent and cutting-edge development. Models like Clip-GCN (Zhou et al., 2024) and GCN+GPT (Chen et al., 2024) have utilized the extensive pre-trained knowledge of LLMs to enrich the text attributes within GCNs, significantly boosting performance in tasks such as fake news detection.

Incorporating GCNs with RNNs, including LSTM and GRU variants, has also proven effective. These architectures benefit from RNNs’ capability to model sequential dependencies, which complements GCNs’ graph-based relational understanding. Hybrid models like GCN-LSTM (Gao et al., 2020) and BiGRU+GCN (Dong et al., 2022) have leveraged this dual capability to capture both long-range dependencies and local contextual information, resulting in improved text classification outcomes.Beyond these combinations, GCNs have been integrated with a variety of other architectures, including CNNs and Transformers, to further enhance their performance. Models like GCN-CNN (Zeng et al., 2022) and GTG (Liu et al., 2023) benefit from CNNs’ ability to capture local features and Transformers’ capacity for contextual understanding, resulting in robust and accurate text classification systems. From early GCN-BERT combinations to the latest GCN-LLM integrations, the augmentation of GCNs with generative models has consistently driven advancements in text classification. By leveraging the complementary strengths of GCNs and various generative models, researchers have developed increasingly sophisticated and effective text classification techniques, leading to continuous improvements in performance across a wide range of applications.

In supervised text classification, the GCN model is trained using labeled data. More specifically, nodes of documents that form the training set make use of associated labels from a set of one or more predefined classes to train the model such that it can predict labels for unseen documents. During training, the model’s parameters are optimized using a loss function, such as Binary or Categorical Cross-Entropy, which measures the difference between the model’s predicted outputs and the true labels of the training examples.

The following approaches aim to build upon TextGCN in a multitude of ways, i.e., optimizing the architecture for improved efficiency, stacking multiple graphs to capture additional context, or addressing its limitations so it may effectively perform in inductive settings.

In a semi-supervised setting, the model is trained using a combination of labeled and unlabeled data. The labeled data guides learning of the relationship between inputs and outputs as it does in supervised learning. However, semi-supervised approaches additionally use unlabeled data to improve the quality of the learned representation so that it is more robust and better generalizes. In practice, this can be achieved by minimizing a loss function that combines the classification loss for the labeled examples with a regularization term that encourages the model to learn similar representations for similar documents.

In self-supervised conditions, the model is able to learn a meaningful representation of input documents without any explicit supervision through training labels. Instead, the model is trained to predict certain hidden features of the input document based on some other unhidden aspect of the same document. More specifically, the model solves a pretext task from unlabeled data, such as predicting the next sentence in a document or reconstructing a corrupted version of the same document. By doing so, the model can learn to capture important semantic and syntactic features of the documents, which can be useful for downstream tasks such as text classification.

Weak supervision is a form of machine learning where the model is trained using limited, or imprecise labels. Instead of having access to a fully labeled dataset, weak supervision leverages various forms of indirect supervision to construct approximate labels. These forms can include domain knowledge, heuristics, and other semi-automated methods. Weakly supervision is particularly useful in scenarios where obtaining fully labeled data is challenging, such as in large-scale datasets or specialized domains like natural language processing.

The landscape of GCN approaches for text classification has evolved significantly, marked by a transition from traditional supervised methods to more sophisticated semi-supervised and self-supervised techniques. Initially, supervised approaches dominated the field, focusing on leveraging labeled data to train models. These early models, such as TextGCN (Yao et al., 2019), TL-GNN (Huang et al., 2019) and SGC (Wu et al., 2019), aimed to optimize computational efficiency and reduce complexity while enhancing classification performance.

As the field progressed, researchers explored multigraph approaches like TensorGCN (Liu et al., 2020), which utilized multiple graphs to capture richer contextual information, such as semantic and syntactic structures. This shift allowed models to better understand nuanced text relationships, leading to improved classification accuracy. Inductive approaches such as TextING (Zhang et al., 2020) and InducT-GCN (Wang et al., 2022a) also emerged to address the limitations of transductive methods, enhancing the models’ ability to generalize to unseen data without retraining. The development of multilabel classification techniques, including HBLA (Cai et al., 2020) and GCN-BERT (Liu, 2022), marked another significant advancement. These models handled the complexity of assigning multiple labels to a single instance, addressing challenges like label correlation and class imbalance. Recent innovations introduced specialized subcategories, like multilingual models (e.g., MSA-GCN (Mercha et al., 2024)) and hierarchical models (e.g., AMKI-HTC (Feng et al., 2024)), expanding the scope of GCN applications to more complex and varied classification tasks.

In subsequent years, there was a noticeable shift towards semi-supervised methods, motivated by the need to leverage large amounts of unlabeled data, which is often more readily available than labeled datasets. Notably, HeteGCN (Ragesh et al., 2021) demonstrated consistent high performance across various datasets, solidifying the importance of semi-supervised approaches in achieving robust and scalable text classification. More recently, the research focus has shifted towards self-supervised learning, reflecting a broader trend in machine learning towards minimizing the dependency on labeled data. Advanced architectures like Cont-GCN-BERT (Wu et al., 2023) represent the cutting edge research, utilizing self-generated supervisory signals to train models. These methods have shown remarkable performance by integrating self-supervised learning with powerful pre-trained language models.

Overall, there is a consistent trend of performance improvement across benchmarks. Early models focused on addressing basic computational and memory constraints, while later models incorporated more complex architectural innovations to capture richer contextual information and handle diverse classification challenges effectively. The semi-supervised and self-supervised approaches reflect a growing emphasis on scalability and adaptability, allowing GCNs to be applied to larger and more varied datasets without relying on labels.

In this section, we provide specifics on some of the most widely used datasets in relevant literature that have been used to benchmark the text classification performance of GCN methods. As results on these datasets have been reported across numerous studies, we can use them to provide a more meaningful comparison of approaches to highlight their relative strengths and limitations. Their statistics in the standard configuration (Yao et al., 2019) are also summarized in Table 3.

• •

  20 NG: The 20 NG dataset contains 18,846 newsgroup documents evenly categorized into 20 different categories, covering a broad spectrum of topics such as sports, politics, technology, and religion, among others. In total, 11,314 documents are in the training set and 7,532 documents are in the test set. In addition to text classification, this dataset has also been used for text clustering and out-of-distribution detection.

• •

  Reuters: This dataset is a collection of documents that appeared on Reuters newswire in 1987 and has been widely used for text classification. R8 and R52 are popular subsets of the Reuters dataset. The former has 8 news categories split into 5,485 training and 2,189 test documents while the latter contains 52 categories, split into 6,532 training and 2,568 test documents.

• •

  Movie Review (MR): The MR dataset is comprised of movie reviews and used primarily for sentiment analysis, i.e., whether a review is negative or positive. It contains 5,331 positive and 5,331 negative reviews.

• •

  Ohsumed: Ohsumed collects medical abstracts tagged by one or multiple classes from 23 cardiovascular disease categories from the MEDLINE database. Since most literature focuses on single-class classification, out of these only 7,400 single-category documents are retained, out of which 3,357 make up the training set and 4,043 documents are in the test set.

• •

  CoLA: The CoLA (Corpus of Linguistic Acceptability) dataset is a collection of English sentences labeled for grammatical acceptability. The labels are binary (i.e., a sentence is either linguistically acceptable or it is not). This dataset was introduced by Warstadt et al. (Warstadt et al., 2019) and consists of 10,000 sentences from various sources, including linguistic literature, standardized tests, and online forums. The publically available version has 9594 sentences in training and development sets and 1063 sentences in the test set.

• •

  SST-2: First introduced by Socher et al. (Socher et al., 2013), this variant of the Stanford Sentiment Treebank (SST) dataset is a collection of movie reviews labeled with binary sentiment classification (positive or negative). The reviews are parsed into constituency trees and labeled with sentiment annotations for each sub-phrase, allowing for fine-grained analysis of the sentiment of the text. The dataset consists of over 11,000 sentences from movie reviews and is widely used as a benchmark dataset for evaluating the performance of models on text classification tasks.

These datasets have been used as benchmarks for a plethora of applications in addition to text classification. Some of these have been summarized in Table 4.

This section serves as a primer for various metrics that have been used for text classification in literature. In the definitions that follow $TP$, $FP$, $TN$, $FN$ have been used to denote true positive, false positive, true negative, and false negative, respectively: Accuracy = (TP+TN)/(TP+FP+FN+TN), Precision = TP/(TP+FP), Recall = TP/(TP+FN), F1-score = (2.Recall.Precision)/(Recall + Precision). In literature, primarily the test accuracy has been used to evaluate the performance of various models. However, F1-score, Precision, and Recall are also used to evaluate an approach depending on the exact nature of the application and distribution of a dataset.

In general, accuracy is a good measure when we have a balanced class distribution, whereas the F1-score is usually best for imbalanced classes. Precision is preferable when the goal is to optimize for True Positives while one should choose Recall if the occurrence of false positives is more desirable than the occurrence of false negatives. The F1-score represents a balance of sorts between Precision and Recall as it is derived from their harmonic mean. For multi-class classification problems, most text classification methods compute precision, recall, or F1-score for each class and compute its macro average. Such averaging considers each class equally important and balances out the effect of majority classes.

This section compares the performance of various methods discussed in this review over a range of benchmark datasets. Primarily, comparisons have been made by evaluating approaches within each category, namely supervised, semi-supervised, and self-supervised. However, cross-category comparisons have also been presented for a more thorough understanding of the relative strengths and weaknesses of these approaches. While accuracy scores are the most abundantly reported metric across all of these approaches, macro-averaged F1-scores have also been considered wherever possible to highlight any notable trends across different categories. Metrics for all GCN methods have been categorically presented under different conditions in Tables 5 to 7. For each of these tables, if a set of results has been obtained under a different train/test split than in (Yao et al., 2019), its details can be found in the table notes. If for a given split, results have been obtained from papers other than the original paper of that approach, those have also been cited next to the method name.

For text classification, Deferrard et al. (Defferrard et al., 2016) applied their approach to the 20 NG dataset. While it was only able to attain second place next to the multinomial Naive Bayes classifier with an accuracy of 68.3, it outperformed traditional fully connected networks while having fewer parameters. This approach was later formalized and further generalized for improved scalability and classification performance in large-scale networks by (Kipf and Welling, 2016a). (Yao et al., 2019) focused exclusively on text classification and proposed a GCN based on (Kipf and Welling, 2016a) and formulated using a novel heterogeneous graph approach. Experiments on 20 NG, R8, R52, MR, and Ohsumed demonstrated that their proposed method was able to outperform existing state-of-the-art solutions without relying on any pre-trained word embeddings, especially when training data was scarce. This would provide the basis and serve as a frequent benchmark for all subsequent approaches. Works that immediately followed, such as (Huang et al., 2019) and (Wu et al., 2019), sought to optimize the approach in (Yao et al., 2019), and while there was only an incremental improvement in classification performance, it was clear that proposing more computationally efficient, scalable, and robust models was their primary goal. Later, (Zhu and Koniusz, 2020) additionally optimized this approach for handling over-smoothing due to a higher number of graph convolutions using the Markov Diffusion Kernel, while more recently, (Wang et al., 2023a) opted for a different route by addressing local intra-class diversity and local inter-class similarity that are implicitly encoded within the graph structure. In general, these approaches improved TextGCN performance in different ways without relying on pre-trained embeddings or any extrinsic knowledge sources. These are, however, primarily transductive in nature. In terms of test accuracy, the most notable improvements in classification performance came when researchers attempted to rework (Yao et al., 2019) by either enriching the graph representation to capture more textual context (Liu et al., 2020) or by augmenting the GCN with other models/embeddings such as BERT (Lu et al., 2020), (Lin et al., 2021) to achieve a solution that was, in theory, greater than the sum of its parts. Among these divergent approaches, the latter yielded the most promising results. In this regard, (Lin et al., 2021) has generally attained SOTA performance across multiple datasets (Table 5).

Another line of research attempted to extend the existing transductive GCN-based approaches to inductive settings to allow for online testing, that is to generalize patterns and relationships from the training data to make accurate predictions on new, unseen instances (Zhang et al., 2020), (Wang et al., 2022a). These methods were able to demonstrably outperform TextGCN under inductive constraints such as limited labeled data. InducT-GCN, in particular, bested various GCN-based baselines and certain models using pre-trained embeddings owing to its better generalization capabilities. Similar to inductive approaches, semi-supervised approaches also demonstrate their efficacy with limited labeled datasets as they are able to compensate for this constraint by extracting and supplementing additional information from unlabeled data and improve model performance. In literature, researchers have employed a variety of subsetting and sampling techniques to demonstrate the effectiveness of their proposed models in limited labeled settings. While this makes it difficult to draw a definitive conclusion about the overall state of semi-supervised approaches, we can still gain valuable insights by considering common baselines.

As graph convolutions enable information sharing among neighboring nodes through the inherent message propagation mechanism, GCN models can transfer knowledge from labeled nodes to unlabeled ones and essentially leverage the underlying graph structure to improve their ability to make predictions on the unlabeled data. While this was successfully demonstrated early on for citation networks in (Kipf and Welling, 2016a), semi-supervised GCN approaches for modeling free text are fairly recent. The authors of (Li et al., 2021) claimed their approach to be the first to model free text under strict semi-supervised conditions and demonstrated their model’s performance gain over (Yao et al., 2019) and (Liu et al., 2020) when using 20 labeled samples per class for training. Different variations of (Ragesh et al., 2021) were also able to outperform (Yao et al., 2019) across the board with regards to test accuracy and f1 scores, albeit under different limited labeled data conditions (20% stratified sample of training documents as in (Yao et al., 2019)). However, the performance gains were not as pronounced in the large labeled data scenario. The same was also true for the approach in (Zhao et al., 2022), which reported slight gains over (Ragesh et al., 2021) under the same large labeled conditions. (Yang et al., 2021b), (Wang et al., 2022b), and (Cui et al., 2022) also exhibited improved performance over (Yao et al., 2019) across multiple datasets under their respective limited labeled conditions (Table 6). Self-supervised GCN techniques for text classification have also begun to gain prominence in recent years. Like semi-supervised approaches, these are also able to leverage unlabeled data to improve model performance. (Yang et al., 2022) was able to attain performance comparable to the best supervised approach that did not use pre-trained embeddings in the large labeled setting and that of (Yang et al., 2021b) in the same limited labeled setting. Moreover, (Wu et al., 2023) utilized various pre-trained language models along with their proposed ATAD scheme and reportedly outperformed all aforementioned methods in offline settings while also faring similarly well in online settings (Table 7). However, it should be noted that this approach is very recent and requires further validation and scrutiny by the research community at large. Nonetheless, it offers valuable insight into the trajectory of research in this domain and could be a promising avenue for future exploration.