Detect All Abuse! Toward Universal Abusive Language Detection Models

We briefly review the seven ALD benchmark datasets (Table 1), which were collected from different online community sources and focused on multiple compositions. Waseem [Waseem and Hovy, 2016] is a Twitter ALD dataset regarding the specific aspects of racist and sexist. The collected tweets were labeled into Racism, Sexism or None. HatEval [Basile et al., 2019] is a Twitter-based hate speech detection dataset released in SemEval-2019. It provides a general-level hate speech annotation, Hateful or Non-hateful, especially against immigrants and women. OffEval [Zampieri et al., 2019] covers the Twitter-based offensive language detection task in SemEval-2019. It annotates as Offensive or Not-offensive, and includes insults, threats, and any form of untargeted profanity. Davids [Davidson et al., 2017] is a Twitter-based ALD dataset, which includes three classes, Hate, Offensive or Neither based on the hate speech lexicon from Hatebase.org. Founta [Djouvas et al., 2018] is a large Twitter-based ALD dataset claimed to be annotated with high accuracy based on their proposed incremental and iterative annotation method. It is annotated with four classes, Hateful, Abusive, Normal or Spam. FNUC [Gao and Huang, 2017] is a hate speech detection dataset, which was collected from complete Fox News discussion threads, and annotated with the general level categories Hateful or Non-hateful. StormW[de Gibert et al., 2018] is a Stormfront-based hate speech detection dataset with general-level labels Hate and NoHate. Stormfront is a supremacist forum where people promote white nationalism and antisemitism.

In the early stages, ALD was commonly addressed via hand-crafted rules and manual feature engineering. The first reported ALD work[Spertus, 1997] utilised a decision tree to detect hostile messages based on heuristic rules. ?) and ?) added lexicon-based features together with semantic rules and designed a linear SVM and Naïve Bayes classifier for detecting hostile language. ?) first applied in ALD neural networks with the paragraph2vec [Le and Mikolov, 2014] representation. ?) introduced a Yahoo! dataset and tested it with neural networks by applying a combination of word, character-based and syntactic features. Recently, deep learning techniques have become popular in ALD. ?) tested FaxtText/Glove, Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTMs) in detecting hate speech. ?) designed a HybridCNN (word-level and character-level) model on abusive tweet detection in both one-step and two-step style. Several works have applied bidirectional Gated Recurrent Unit (Bi-GRU) networks with Latent Topic Clustering (LTC) ?) and a a transformer-based framework ?). Some works integrated user profiling into their ALD models. ?) utilised the bi-LSTM to model the historical behaviour of users to generate inter-user and intra-user representation. ?) applied node2vec [Grover and Leskovec, 2016] to the constructed community graph of users to derive the user embedding. However, a user profiling-based approach is only possible when the user profiles are public and when the domain provides the user-community relation information.

In the Cross-Attention Gate Flow, first, we use a cross transformer encoder for refining our four types of embedding: Directed abuse embedding $D$, Generalized abuse embedding $G$, Explicit abuse embedding $E$ and Implicit abuse embedding $I$. Before putting them into the cross transformer encoders, we combine $D$ with $G$ as Target embedding $T_{e}$ and broadcast $I$ to sequence length $N$, them combine it with $E$ as Content embedding $C_{e}$. Normally, for the transformer encoder [Vaswani et al., 2017], the attention is calculated using key ($K$ of dimension $d_{k}$), query ($Q$), value ($V$):

$$\displaystyle Attention(Q,K,V)=softmax(\frac{QK^{T}}{\sqrt{d_{k}}})V.$$

(1)

We call the cross transformer here Cross at Beginning(CB). Similar to the original transformer encoder, each encoder contains one or more encoder stack(s), which mainly consists of two sub-layers: a multi-head attention layer and a fully connected feed-forward neural network (FNN). A residual connection followed by layer normalization is employed around each of the two sub-layers before feeding to the next sub-layer. Another way to produce the joint integration occurs before the FNN layer. The output of Multi-Head Attention will be the input for the FNN layer, and then an Add & Norm layer is applied. Normally, the output of transformer encoder is calculated by

$$\displaystyle Output=norm(FNN(O_{MHA})+O_{MHA})$$

(3)

The input for FNN can also be switched for Content and Target, which is called Cross in the Middle (CM), the output of transformer encoder will be calculated by

$$T_{h}=norm(FNN(C_{MHA})+T_{MHA}),\quad C_{h}=norm(FNN(T_{MHA})+C_{MHA})$$

(4)

If the cross happens both at the beginning and in the middle, the structure will be called Cross at the Beginning and in the Middle (CBM). The comparison of different cross transformer structures will be discussed in 5.2. Both of the input embeddings $T_{e}$ and $C_{e}$ are of shape [$N$, $D_{e}$], where $D_{e}$ is the sum of the dimension of the concatenated embedding. The transformer encoder will output $T_{h}$ and $C_{h}$ in the same shape [$N$, $D_{e}$]. The hidden state of encoders $T_{h}$ from $T_{e}$ and $C_{h}$ from $C_{e}$ will be used to compute the initial abusive language probability, which is the major input of our bi-directional aspect gate flow.

On top of the Cross-Attention, we introduce the Bi-directional Aspect Gate Flow that contains two gates: content gate and target gate. Denote the input sequences to our gates from the previous layer encoder as $T_{h}\in\mathbb{R}^{N\times D_{T}}$ and $C_{h}\in\mathbb{R}^{N\times D_{C}}$ where $N$ is the sequence length while $D_{T}$ and $D_{C}$ equal to dimension of target embedding and content embedding respectively. In the content gate, we first flatten $T_{h}$ to be $T_{hf}\in\mathbb{R}^{1\times(N*D_{T})}$. We then pass $T_{hf}$ through a dense layer and apply the $Softmax$ function. The resultant $P_{Th}$ is a $D$-dimensional probability vector, where $D=N_{cls}$ is the number of distinct labels to classify, $W_{C}\in\mathbb{R}^{D\times D_{C}}$ is the weight matrix and $b_{C}\in\mathbb{R}^{1\times D}$ is the bias vector. Then we broadcast $P_{Th}$ over $N$ tokens. This yields $\hat{P_{Th}}\in\mathbb{R}^{N\times D}$. Then we concatenate $\hat{P_{Th}}$ with transformer encoder output state $C_{h}$ from content source, generating the augmented content state $O_{C}\in\mathbb{R}^{N\times(D+D_{C})}$. We then again flatten $O_{C}$ and pass the output to the dense layer, producing an output matrix $P_{C}\in\mathbb{R}^{1\times D}$.

The procedure in the target gate is almost the same as the content gate. Here we flattened the input sequence $C_{h}$, generating the flattened output $C_{hf}\in\mathbb{R}^{1\times(N*D_{C})}$. We then pass the result through a dense layer and apply the $Softmax$ function. The resultant $P_{Ch}$ is also broadcast to be $\hat{P_{Ch}}$ and then concatenated with the target encoder output state $T_{h}$, where $O_{T}\in\mathbb{R}^{N\times(D+D_{T})}$ is the augmented target state as output matrix. Finally, $O_{T}$ is also flattened and then passed to the dense layer, which produces the output matrix $P_{T}\in\mathbb{R}^{1\times D}$.

We propose a hierarchical fusion, which fuses linguistic behaviour outputs ($P_{G}$) with content gate output ($P_{C}$) and target gate output ($P_{T}$) respectively and uses two CNNs to integrate that fusion to get $C_{C}$ and $C_{T}$, then we concatenate $C_{C}$ and $C_{T}$ then flatten it to $F_{F}$. Finally, a multi-layer perceptron (MLP) is used for final prediction:

$$L_{1}=ReLU(W_{1}\cdot F_{F}+b_{1}),\quad L_{2}=ReLU(W_{2}\cdot L_{1}+b_{2}),\quad Z=softmax(W_{3}\cdot L_{2}+b_{3})$$

(5)

Three layers are stacked. For the each layer, $W_{i}$ and $b_{i}$ represent the weight matrix and bias vector, and the ReLU activation function is used for the first two layers. For the last layer, to get the probability of each class $Z$, softmax layer is used.

In this part, we compare our model with six baseline models over all seven datasets, discussed in Sec 2.1. These baseline models are constructed with various word representations as well as different neural networks or classifiers. Table 2 presents the weighted average f1 performance of each baseline model and our model over each dataset. Our model outperforms the baseline models for all these seven datasets. Applying multiple aspect embeddings enables our model to process the texts from multi-perspective views. The Cross-Attention gate flow makes it possible to obtain the mutual enhancement between the two different aspects. Although some of the baseline models such as OTH, MFR also combine two embedding approaches (Chars2vec and Glove) to get more information, they still just consider the general information of the texts rather than extract information in a targeted fashion from various aspects. For these reasons our model can achieve performance above the baseline models.

As well as comparing our model with the baseline models, we also make some observations from comparing the six baseline models amongst themselves. Firstly, OTH and MFR use the combined embeddings of Chars2vec and Glove which gives more information. So, they can achieve relatively better weighted average f1 scores compared to most other baseline models which just use a single embedding method. Secondly, the results of TWL and LTC indicate that the bi-directional recurrent neural network leads to better performance than the simple forward recurrent neural network. This means that not only the future states but also the past ones will affect the prediction results. Thirdly, although we may not consider TF-IDF with SVM to be as good as Chars2vec or Glove with deep neural networks, TIS baseline model never gets the worst weighted f1 score for the seven datasets when compared with other models. In fact it even outperforms other baseline models on Waseem and Founta. For both datasets, there might be some particular words which are really significant for identifying the class. So TF-IDF can achieve good results for these two datasets.

Next, an analysis of examples to show how different models perform on different datasets is conducted, and the results are shown in Table 3. The first example is from Waseem, “Really bitch really? Get on the runway and back in the kitchen.”, which should be predicted as Sexism. It is quite explicit in that the word “bitch” is in this sentence, and this makes TIS predict it as Sexism easily since TF-IDF is focusing on the word occurrence. Besides, “back in the kitchen” is implicit Sexism, implying women should be in the kitchen. The similar patterns can be found from the second instance “thegeek_chick please answer” by explicitly mentioning the word ‘chick’. The third and fourth samples represent abusive language or hate speech about the topic Feminism. The third explicitly stated the words ‘Feminism’ and ‘Awful’ and TIS and our model successfully detected the abuse with an explicit hate speech aspect identification. Our model, which considers the explicit and implicit aspects, can predict the sentence as Sexism easily. Another example is from FNUC, “BLM is a Hate group Given Birth by Obama’s comments That put a target on the backs of police officers” which should be Hateful. This comment insults the “Black Life Matters” by calling it a Hate Group. Normally, describing something as a hate group is not hate speech, but in this case, calling BLM a hate group is racism. This is not easy for the baseline models to spot, and only our model predicts it correctly. For the last example from StormW, “i am a white nationalist of a christian faith but still am a white nationalist for racial survival the anti racist christians are the true chirstians enemy”, the user described himself as “white nationalist” which is one kind of hate speech, and OTH can predict this sentence as Hate. The reason is that the CNN used in OTH can capture the information for phrases, which is the “white nationalist” here. Besides, our model can predict this sentence correctly since the sentence is a general explicit hate speech.

In this part, three different structures of cross transformer encoders are tested: 1) Cross-transformer at the beginning of the the transformer encoder (CB): exchanging content’s and target’s K and V at the beginning of the transformer encoders as in Figure 2; 2) Cross-transformer in the middle of the transformer encoder (CM): exchanging content’s and target’s input for Feed Forward layer in the transformer encoder, which is in the middle of the transformer encoders; 3) Cross-transformer at both places (CBM): the combination of CB and CM. Due to the poor performance of CM, only results for 7 datasets with CB and CBM structure are shown in Table 4. Besides, to find whether and how GCN is improving the performance of our model, different structures are also compared: 1) Model without GCN; 2) Model with GCN using hierarchical fusion, repeating one or three times. We show one and three times here because on all the datasets our model achieves the best performance with one or three repeated fusions when GCN is also used. Two conclusions are drawn based on the results of CB and CBM:

Firstly, the best model is always the CB model, and the second best is always the CBM model with the same GCN structure. So comparing between CB and CBM structure, CB has a better performance and we use this structure as our final model. Besides, in most cases, CB outperforms CBM if they share the same GCN structure, which also shows that, overall, CBM is worse than CB. Considering the fact that CM is the worst, we can say that cross in the middle transformer encoder will lower the model performance. Exchanging content’s and target’s K,V is important since it allows target aspects to query on the content aspects and vice versa. However, exchanging values before Feed Forward Layer only gives a different add and norm which doesn’t increase the interaction between content aspects and target aspects usefully.

Secondly, our model can have a better performance with GCN when there is user id in the dataset. Not all the datasets provide user id, and as mentioned in Sec 3.1, User Linguistic Behavior embedding is trained by using the user id as the target. For those datasets without userid, the real abusive labels are used as the training target. By comparison, we can find that Waseem, StormW, and FNUC which provide user id in the datasets have a better performance using a model with GCN, and the other four datasets, which don’t provide user id, have a better performance using a model without GCN. Therefore, for the dataset with user id, User Linguistic Behavior which is from GCN, can improve the performance of our model. And for those datasets without user id, the model structure without GCN is recommended.

To check how aspect embeddings contribute to the model, an ablation test on different combinations of the embeddings is conducted on all these seven datasets. We use the CB model without GCN for the prediction. Table 5 presents the weighted average f1 scores for 9 different combinations of four aspect embedding models, including Directed abuse $D$, Generalised abuse $G$, Explicit abuse $E$, and Implicit abuse $I$. Each target and content aspect should include at least one embedding.

For Waseem, the $D+G+E+I$ combination achieves the best performance with the weighted average f1 score 82.35 and most other combinations have a slightly lower performance. In contrast, $D+I$ gets the worst weighted f1 score of 61.93. The reason why $D+I$ is much worse than other combinations may lie in two facts: 1) In this dataset, abusive language is generally more explicit rather than directly aiming at a specific target in an implicit way. 2) Even humans can not distinguish Direct Abuse in an Implicit way easily, and it can be very difficult for the annotators to annotate the label correctly. Besides, the $D+G+E+I$ combination outperforms other cases because it takes all the aspects into consideration. Similar results occur on other Twitter datasets Davids, HatEval, OffEval and Founta, $D+G+E+I$ achieves the best while $D+I$ is much worse. For FNUC, due to the small volume of dataset and imbalanced labels, not all the combinations have a good prediction result. $D+G+E$ having the best performance implies that the dataset doesn’t have a large number of implicit abuse samples. For StormW, $D+G+E+I$ gets the best performance. Besides, $G+E$ also has a good performance. The reason is that this dataset is collected from a racism forum and most hate speech on that website is generally abusive in an explicit way. Based on the analysis of the different embedding combinations on these datasets, we can conclude that the embeddings used may vary based on different kinds of datasets, but combining them all is always a good idea. Although four specific different embeddings are selected in our model to represent four different aspects, other kinds of embeddings could also be used as long as they can represent the corresponding aspects.