Exploring Hate Speech Detection with HateXplain and BERT

Hate speech is defined as a group of words or phrases that are targeted towards a certain group of people and are derogatory in nature. Hate speech can fall under many categories - against certain sexual orientation (homophobic), against certain religions (islamophobic), against certain races (racial slur), etc.

Hate Speech detection is the task of detecting such sentiment from text. The task is to distinguish between normal language, offensive language and hate speech. While the distinction with the first category is fairly straightforward, the distinction among the latter 2 categories is non-trivial. Additionally, in order to make the task more human-detection-like, we also include the target community and span of words that cause the prediction. This makes our problem statement more inclined towards explanability - helping us answer as to why the deep models predict the way they do.

We tackle the problem of hate speech detection by using the world’s first rationale-based dataset on this topic - HateXplain. Our BERT implementations work well with the plethora of metrics available at our disposal in all three categories - accuracy, explainability and bias. Specifically we do the following in our work:

• •

  $\lambda$ (attention rational importance) tuning on the BERT model. The hypothesis of the authors is tested to see if more attention to rationales improves the performance.

• •

  We introduce new methods to take the rationale attention itself - conservative attention (all annotators agree), lenient attention (anyone agrees), and normal attention (mean attention).

• •

  We also use masking of the target community names to improve on the bias metrics and meet the expectations of our hypothesis, albeit at a very minor cost of overall accuracy (which was also expected due to loss of information)

• •

  Finally we conclude with an extensive error analysis that is interesting in terms of the improvements on different fronts. We visually see the improvements in class-wise prediction and a reduction in marginal community missclassification.

The HateXplain dataset by Mathew et al. (2021) is unique in 3 facets or factors.

1. 1.

   It focuses on the category of speech - normal, offensive or hate speech.

2. 2.

   It assigns the targeted community to the post. This community is based on religion, sex, caste, race, geography etc.

3. 3.

   It associates word and phrase level spans of the text with the prediction.

These characteristics are in line with the growing trend and need to make deep learning models more interpretable and explainable. Using the rationale in combination with the prediction class and target community is also expected to make the model predictions more human-like.

Another unique aspect about the dataset is the distinction it makes between offensive and hate speech by using the standard set by Davidson et al. (2017). This is extremely relevant in today’s time and age when there is immense offensive content on the internet, not all of which comes under the category of hate speech because it is not targeted to be derogatory towards a particular group. Examples of such instances cited by Davidson et. al. in Davidson et al. (2017) include the use of ’n*gga’ by the African American Community, the use of ’b*tch’ and ’wh*re’ in rap lyrics etc.

Normal text often misclassified as offensive by BERT-like models - full of words that are usually offensive.
Anyone leaked the info on who at youtube is the white supremacist neonazi fiddling with the algo seems that should drop next right along with the metrics linking up the infosec talking heads with their harassment and nazi socks right guise.

The annotation has been done using Amazon MTurk and the selected annotators are all given batch-wise posts to annotate in several stages as highlighted by Mathew et al. (2021). The target label as well as the targeted class are decided upon majority voting. The annotations are given by annotators as boolean vectors equal to the length of the sentence. They are averaged, and then a softmax is applied to make the probability uniform and normalized. This acts as a ground truth attention, deactivated if the label is ’normal’ speech.

Comprises of metrics such as Accuracy. AUROC and F1-score. These reflect the statistical traditional performance of the model over the data.

The HateXplain paper explains that there need to be metrics to measure the unintended bias on certain words and classes. As per the standard and metrics set by Borkan et al. (2019), we use multiple metrics.

A higher value of Subgroup AUC means that the model is doing a good job at distinguishing the toxic and normal posts specific to the community. A higher value of Background Positive Subgroup Negative (BPSN) AUC. means that a model is less likely to confuse between the normal post that mentions the community with a toxic post that does not. A higher value of Background Negative Subgroup Positive (BNSP) AUC means that the model is less likely to confuse between a toxic post that mentions the community with a normal post without one.

Plausibility refers to how convincing the interpretation is to humans, while faithfulness refers to how accurately it reflects the true reasoning process of the model. To measure the plausibility, we consider metrics for both discrete and soft selection - the IOU F1- Score and token F1-Score metric for the discrete case, and the AUPRC score for soft token selection. The inspiration is taken from DeYoung et al. (2019). A high value of comprehensiveness implies that the rationales were influential in the prediction. Sufficiency measures the degree to which extracted rationales are adequate for a model to make a prediction.

The model is interpreted in two steps. Firstly, we calculate $L_{pred}$ that is the loss on the predicted class. Secondly, the BERT model has an extra fully connected layer at the end. We try to match the attention values of the CLS token in this last layer to the ground truth attention and use a cross entropy loss to do so. This gives us $L_{att}$. The $\lambda$ value controls the trade off between the two.

Without a doubt, the most important hyperparameter in our BERT model was the $\lambda$ value, which regularizes how much effect the attention loss has on the total loss. We have reported the efficacy of tuning the hyperparameter $\lambda$ for three different classes of metrics:

1. 1.

   Performance Metrics

2. 2.

   Bias Metrics

3. 3.

   Explainability Metrics

We observe that the performance of the model improves (in general) with increase in $\lambda$. These trends are observable from Table 1. This was in sync with our hypothesis, because hate speech detection is a contextual task, that should improve when relevant words are better attended to. In fact, this was the reason why bag of words-like models were not performing very well on this task. Table 2 reports the values of the bias metrics (Subgroup AUC, BPSN and BNSP) for different values of $\lambda$. We observed that all three metrics were higher at lower values of $\lambda$, as opposed to the performance metrics. Finally, we have tabulated the effect of $\lambda$ on the explainability of the model in Table 3. We observed that the with the exception of Comprehensiveness and Sufficiency, the Explainability of the model is generally better at higher values of $\lambda$. However, in case of Comprehensiveness and Sufficiency, the best values were obtained for $\lambda=1$.

Currently, the ground truth attention is calculated by taking an average of the attention from all three annotators, followed by a softmax function to convert it to probabilities. This gives equal importance to all annotators. Since different annotators come from different backgrounds and perceptions, a sensitive task like hate speech detection can benefit from a non-equal importance. We thus divide experimentation into 3 approaches:

Normal Attention: This is the original papers method as explained above.

Conservative Attention: We attend to a word only if all annotators think it is important. This is analogous to an AND operator. We hypothesize that this change should perform better as the attended will be stronger than the normal mode.

Lenient Attention: We give importance to a word if any annotator thinks it is important. This is analogous to an OR operator. We hypothesize that this would be similar or lesser than the normal attention as we attend to all words with greater leniency.

Improvements and Intuition: We notice from Table 4 that we indeed gain around 1.1% accuracy by using conservative attention, thus confirming our hypothesis. The intuition behind this increase is that the BERT model’s last layer attends to more strongly prediction-correlated words when we we attend only to words marked by all annotators - thus removing bias from the annotations and model.

Clearly, from Table 5 we can see that we were correct. We have relatively stable and high values from bias metrics in case of conservative attention, and massive improvements in case of explainability metrics, with the highest being for conservative attention with $\lambda=1$. Another interesting observation is the fact that lenient attention performs better on explanability metrics, because the active attention weights most definitely capture all words important for prediction. Hence, despite not seeing an accuracy boost like conservative attention, lenient attention weights contribute more to human interpretability, as reflected by high plausibility values.

We noticed in the original implementation that there are many instances of erroneous classification whenever the model comes across offensive words. These are the cases where the model classified the text as hatespeech or offensive just because they found a word belonging to a targeted community even though the statement did not convey any hate and was in fact a normal sentence (eg. rap songs with the word n*gga). We have attempted to reduce these kind of misclassifications by masking certain key words from these targeted communities during the training of the model. The intuition behind this is that if the model tries to judge a sentence just by the context and not the community then it wouldn’t normal texts with with certain key words as toxic during testing. Masking the community names/target names on one hand should give the model an independence to classify based on non-community words, but it will lead to loss of information. As is visible from Table 7 and Table 6, the former reason boosts our explainability and bias metrics, while the latter slightly reduces our accuracy. Overall, the model performed as expected and was indeed successful in removing bias and increasing explainability.

We found that masking target communities in the training data, despite increasing the accuracy slightly, leads to a reduction in the bias. We believe this is because the model might become more susceptible to classifying normal sentences as toxic, since it can be triggered by rationales such as ”get rid of”, ”erased”, ”absolutely useless”. While such terms - when used in the context of a target community - can constitute hatespeech, they can also be classified as normal speech if one were to talk about ”getting rid of” trash. Another example would be [UNK] is extremely irritating, we must get rid of them. The same sentence would be considered toxic if the [UNK] token were to be a target community such as ”Jews”, ”Muslims”, ”Blacks”, etc., but would be a normal sentence if [UNK] was ”Advertisements”,”Mosquitoes”, etc. Given that Subgroup AUC, BNSP and BPSN are based on how well the model is able to identify toxic comments from a subgroup as compared to the background (all other communities combined), we believe these metrics may be negatively impacted since the model may incorrectly label normal sentences as toxic (more false positives).

In order to understand the impact on community-specific performance, we plot the top 10 most misclassified communities in Figure 1. We see significant improvements and progress in reducing misclassifications on highly marginalised and targeted communities using both masking and conservative attentions. As is clearly visible from Figure 1, both of our proposed novelties reduce the misclassifications on highly missclassified and marginalised/targeted communities like African, women, and miscellaneous (other). We also observe some mixed trends in communities like Jewish, where conservative attention does very well due to strict control over rationales, but community masking does not (due to loss of information as highlighted before).

From Figure 2 we can see that conservative attention is able to effectively tackle normal speech that was earlier classified as hate speech or offensive speech, because of its inherent stricter and conservative nature. There is however, minor increase in offensive speech classified as normal speech too. Overall, the confusion matrix of conservative attention is better than the masking and original implementations. A cumulative visual of our observations is given in Figure 3, which clearly demonstrates that we have been successful in reducing the miss-classification of normal text as offensive or hate speech.