TieFake: Title-Text Similarity and Emotion-Aware Fake News Detection ^††thanks: † Corresponding Author: Zhao Kang, [email protected]

Most existing work on fake news detection task treats it as a binary classification problem. Early methods [18, 19] design plenty of hand-crafted features to debunk fake news. These methods train a fake news classifier using text content features. Although these manually chosen features improve the performance of fake news detection, these approaches typically require extensive preprocessing and feature engineering. Recognizing and detecting fake news has become more complex as social media information has exploded in popularity. Researchers have put forth various practical methods [18, 15, 20, 5, 13], which can be briefly reviewed from single-modal (e.g., text or image) and multi-modal perspectives.

Existing methods[18, 21, 20, 5] for single-modal analysis mainly concentrate on extracting textual or visual elements from the news’ text content or image. For example, Yu et al. [22] use convolutional neural networks to obtain high-level interactions and critical features of related news. Recurrent neural networks are used by Ma et al. [19] to learn latent properties from the relevant textual news. In  [23], the authors only exploit the rich visual information with different pixel domains and utilize a multi-domain visual neural network to detect fake news. However, social media platforms offer a wealth of multi-modal data (e.g., images, texts, and videos)[24, 25] that can complement each other and contribute to social media analysis[26, 27].

Researchers realize that multi-modal fusion features may be crucial in identifying fake news because of deep neural networks’ enormous success in learning picture and word representations. Multi-modality fake news detection has recently drawn a lot of interest. Several approaches [15, 28, 29, 30, 31, 32] conduct fake news detection based on multimedia content and obtain superior performance. Jin et al.[15] propose a multi-modality-based fake news detection model, which extracts the multi-modality information, including visual, textual, and social context features, and then fuses them by attention mechanism. Khattar et al.[28] propose a multimodal variational autoencoder that learns a shared representation of the text and images. Shivangi et al.[29] make use of the pre-trained BERT to learn text features and apply VGG-19 pre-trained on the ImageNet dataset to learn image features. Wang et al.[31] propose a novel knowledge-driven multimodal graph convolutional network to jointly model textual information, knowledge concepts, and visual information into a unified framework for fake news detection.

Although most existing approaches show promising performance on fake news detection task, they still fail to fully exploit the data. In this paper, we propose to take advantage of the author’s potential emotion and the similarity between the title and text.

Attention mechanisms have been shown to be effective in various tasks such as image captioning [33, 34], machine translation [35] and recommendation system[36]. Bahdanau et al. [35] firstly introduce attention to the machine translation task to allow the model to automatically search for parts of a source sentence that are relevant to predicting a target word. Soon after, Transformer[37] is proposed to solve the sequence-to-sequence problem, replace LSTM with an attention structure, and achieve a state-of-the-art quality score on the neural machine translation task. Recently, attention mechanisms have been incorporated into fake news detection methods. For example, Chen et al.[38] propose a deep attention model based on recurrent neural networks(RNN) to learn selectively temporal hidden representations of sequential posts for identifying fake news. Motivated by the successful applications of the attention mechanism, we introduce a scaled dot-product attention mechanism on title features and textual features to compute the similarity between news title and text.

To precisely utilize the semantic meaning of the word and the linguistic contexts, we employ a 12 encoding layers version of Bidirectional Encoder Representations from Transformers (BERT [39]), which takes a sequence of words as input. We model a given text content $t$ as the textual embedding $R_{T_{bert}}$ = $\{\mathbf{T}_{1},\mathbf{T}_{2},\cdots,\mathbf{T}_{m}\}$ (where $m$ denotes the number of words) from the second last output layer of the module, and passes the embedding through a fully-connected layer to reduce down to the final dimension of length 32, i.e., $R_{T}\in\mathbb{R}^{32}$. The operation of the fully-connected layer in the textual feature extractor can be represented as:

where $\sigma$ is the ReLU activation function and $W_{vt}$ is the weight matrix of the fully connected layer. Similarly, we can use the same method to generate the feature vector of the title $R_{Ti}\in\mathbb{R}^{32}$.

Given an image $V$ attached to a news text, we employ the pre-trained ResNeSt-50 [40]. On top of the last layer of ResNeSt-50, we add a fully connected layer to adjust the dimension of final visual feature representation to length 30, i.e., $R_{V}\in\mathbb{R}^{30}$. During the joint training process with the textual feature extractor, the parameters of pre-trained ResNeSt-50 are kept static to avoid overfitting. The operation of the last layer in the visual feature extractor can be represented as:

where $R_{V_{resnest}}$ is the visual feature representation obtained from pre-trained ResNeSt-50 and $W_{vf}$ is the weight matrix of the fully connected layer in the visual feature extractor.

To comprehensively represent emotion, we employ publisher emotion extractor [41] to obtain a variety of features from news contents, including the emotion category, emotional lexicon, emotional intensity, sentiment score, and other auxiliary features. The corresponding feature representations are denoted by $R_{E}^{cate}$, $R_{E}^{lex}$, $R_{E}^{int}$, $R_{E}^{senti}$, and $R_{E}^{aux}$, respectively. Among them, emotion category, emotional intensity, and sentiment score provide the overall information, while the other two provide word- and symbol-level information. We concatenate all five kinds of features as $R_{E}$ with final dimension of length 38, i.e.,

In this part, we propose to apply the scaled dot-product attention mechanism on title features and textual features to capture how well the title and text are related to each other in the news. We utilize the attention mechanism for the title and textual features in two directions. Figure 4 shows the general design of our proposed attention mechanism.

When we use information from a text to compare to a title or use the text vector representation $R_{T}$ as Query and title features $R_{Ti}$ as Key and Value. Mathematically, the Query, Key, and Value are defined as:

where $d_{Ti}=d=32$, $W_{Q}$, $W_{K}$, $W_{V}$ are weight matrices.

The output matrix of the scaled dot-product attention applied on $Q$, $K$, and $V$ is calculated as:

where $Att_{T\xrightarrow{}Ti}$ is the attention’s output matrix when we use text vector representation $R_{T}$ as Query and title features $R_{Ti}$ as Key and Value.

Similarly, when we use information from the title to make comparison to the text, we obtain $Att_{Ti\xrightarrow{}T}$, in which the title vector representation $R_{Ti}$ is used as Query and $R_{T}$ is used as Key and Value. After obtaining two attention’s output matrices $Att_{T\xrightarrow{}Ti}$ and $Att_{Ti\xrightarrow{}T}$ , we pass them into a fully connected layer with the size of 32, which is the same as $d_{T}$. Finally, we obtain two vectors $R_{T\xrightarrow{}Ti}$ and $R_{Ti\xrightarrow{}T}$.

We deploy a fully connected layer with softmax to predict whether the news articles are fake or real. Above five feature vectors $R_{T}$, $R_{V}$, $R_{E}$, $R_{T\xrightarrow{}Ti}$, and $R_{Ti\xrightarrow{}T}$ are fused together to generate the final news representation denoted as:

where $\oplus$ is the concatenation operator. The fake news detector is built on top of the multi-modal feature extractors and takes $R_{F}$ as input. Finally, given a news article, its label $\hat{y}$ can be predicted by a fully connected layer with all learned parameters.

We compare to the following baselines, which detect fake news using (i) textual (Sec.IV-B1), (ii) visual (Sec.IV-B2), or (iii) multi-modal information (Sec.IV-B3).

Experimental results in Table II further reveal several insightful observations.

(1) In both datasets, multi-domain models work much better than single-domain models. Once again, this validates the benefit of incorporating all kinds of information. For example, additional visual information can be used as complementary information to help detect fake news.

(2) Text-based models work much better than image-based models, demonstrating that textual features could be more helpful than visual ones in detecting fake news. The reason is that it is more challenging to learn the semantic meaning of visual features than textual features.

(3) SAFE outperforms all baselines on the PolitiFact dataset because SAFE jointly uses multi-modal (text and visual) and relational information to learn the representation of news, which is more applicable to small sample datasets. In addition, SpotFake and SpotFake+ achieve better results than other baselines on the GossipCop dataset, indicating that the pre-trained BERT and XLNet can obtain better textual information to improve model performance, which is more applicable to big sample datasets.

(4) The proposed TieFake outperforms all the baselines on both large and small sample datasets. In comparison with SAFE on the PolitiFact dataset, our model improves 4.0% in accuracy, 4.8% in precision, 1.2% in recall and 3.0% in $F_{1}$. In comparison with SpotFake+ on the GossipCop dataset, our model improves 2.8% in accuracy, 2.8% in precision, 2.0% in recall and 2.4% in $F_{1}$. This verifies that the proposed model can jointly capture multi-modal (text and visual) and emotional information, which can better characterize the underlying representation of news content, improving the performance of fake news detection.

The proposed TieFake contains several components, thus we additionally compare the variants of TieFake to show the impact of each component.

• •

  TieFake-T: A variant of TieFake with the textual information is removed;

• •

  TieFake-V: A variant of TieFake with the visual information is removed;

• •

  TieFake-E: A variant of TieFake with the author potential emotion is removed.

• •

  TieFake-S: A variant of TieFake with the similarity between the title and text is removed.

Results in Table  III indicate that (1) integrating news’ textual information, visual information, and author’s potential emotion improves the performance; (2) the proposed method TieFake outperforms TieFake-E, which shows the effectiveness of introducing the potential emotion to our model; (3) the proposed method TieFake outperforms TieFake-S, which proves that mining the similarity between the title and the text is effective for detecting fake news; (4) textual information is much more critical than visual information and potential author emotion.