In-game Toxic Language Detection: Shared Task and Attention Residuals

Toxic behaviour has become a severe problem in recent online games and the gaming industry. The nature of the toxic utterances in the in-game chat is very different from the other online domain, such as social media or online news. It has a much shorter length because game players tend to type in-game chat during playing. The longer utterances occur only in pre- or post-game discussions. With this in-game chat nature in mind, understanding the slot(word token)-level (Weld et al. 2021b) is crucial to detect the in-game toxic language. In this paper, we describe the established slot(token)-based in-game toxic language detection shared task, and present the best model and its novel components. The result shows the novelty of the best model by comparing the baselines from the CONDA (Weld et al. 2021a).

We set up a shared task competition²²2https://www.kaggle.com/competitions/2022-comp5046-a2 for performing a sequence token labelling task to identify the type of semantics conveyed by each slot in-game chat utterances provided by CONDA (Weld et al. 2021a). CONDA consists of 44,869 utterances from chat logs of 1,921 Dota2 matches, with 26,921/8,974/8,974 utterances for training/validation/test sets respectively. CONDA provides 6 distinct slot labels: T (Toxicity), C (Character), D (Dota-specific), S (Game Slang), P (Pronoun) and O (Other) to provide a deeper understanding of game context. A total of 312 student teams participated in our shared task. Each team was allowed 20 submissions per day. In total, we received 3,646 submissions across 4 weeks. Most participants pre-processed the provided data simply by tokenizing the utterances into slots by spaces, as CONDA provides the cleaned utterances to provide slot labels. There are also teams applying data augmentation to increase the number of instances they can have for training. For the input embedding, combinations of syntactic, semantics and domain-related embeddings were included in the participants’ trials. For the syntactic embedding, most teams chose to encode POS tags and/or dependency parsing results based on models provided by SpaCy. For the semantic embedding, participants would either directly use pre-trained GloVE/FastText embedding vectors, or train their own FastText/Word2Vec based on the provided game chat corpus. Some teams tried to find a Dota-related corpus or toxicity detection dataset collected from social media to train the word embedding and inject more domain-related information into the model. A variety of sequence labelling model architectures are explored by the participants, and we will introduce the best-performing team’s method and results.

Our model achieved the best performance from the shared task. It includes Bi-LSTM cells, attention residuals, label forcing and CRF. Since the model uses global information extracted from the attention mechanism as residuals to supplement bi-directional features, it is named Bi-directional Representations with Attention Residuals (BRAR).

Bi-LSTM cells perform feature extraction on the input sequence $x=(x_{1},x_{2},…,x_{t})$ where $t$ denotes the number of tokens. $y=(y_{1},y_{2},…,y_{t})$ is the corresponding slot labels with $c$ unique values. The attention layer aims to understand the global information of the input utterance. Attention $a\in\mathds{R}^{2\lambda}$ is calculated in the following equation where $h_{t}\in\mathds{R}^{2\lambda}$ is the last hidden state of Bi-LSTM cells, $W_{a}\in\mathds{R}^{{2\lambda}\times{t}}$ is a trainable weight matrix and $H\in\mathds{R}^{{t}\times{2\lambda}}$ is the output of Bi-LSTM:

As the global information is not always helpful for understanding each input token, it is treated as the residual of the token-level representation to form the feature $f_{i}\in\mathds{R}^{{t}\times{c}}$, while $\alpha$ is a trainable parameter to scaling and $W_{f}\in\mathds{R}^{{2\lambda}\times{c}}$ is a trainable weight matrix for dimension transformation. When the global information benefits the token-level interpretation, it will speed up the convergence; otherwise, it will not reduce the prediction performance.

The feature $f_{i}$ is enhanced at the label forcing layer to form the emission score, which will be passed to the CRF layer to predict the tag of each token. For example, “gg” stands for “good game”. If all words “gg” in the training set are classified as “S” (Game Slang), the word in the test set will have a high probability of being classified as “S”. In this case, if the probability information of the correspondence between words and labels in the training set can be learned, it will be of great help in predicting the results of the test set. Therefore, the proposed model creatively uses a novel technique: label forcing (LF). Specifically, the label distribution probability of each token in the training corpus is calculated and normalized by the following equation, where $C_{ij}$ is the frequency of that the token $i$ is annotated as the label $j$. The overall architecture can be found in the Figure 1.

The proposed model adopts FastText-50 as the input embeddings. The Bi-LSTM layer contains a hidden size of 5 in each direction, and the layer number is set to 1. The SGD optimizer has been applied with the learning rate of 0.1 and the weight decay of 1e-4. In addition, the training epoch is set to 2 with a batch size of 1 (same as the CONDA). As for evaluation metrics, the overall micro F1 and that for each slot label have been reported on the test set, but the O tag is excluded when calculating the overall F1-Score. The 5 baselines are selected based on CONDA paper. From Table 1 we can conclude that BRAR outperforms other baselines on most F1, especially in predicting T, S, and D labels.

We evaluated the model with different number of Bi-LSTM stacks/layers. The results shown in Table 2 indicate that more layers lead to the lower F1 scores, though there is no significant difference among the three different number of layers. Therefore, we propose to use only 1 Bi-LSTM layer as it is computationally more efficient with a smaller number of parameters.

In this paper, we established a shared task for slot (word token)-based in-game toxic language detection, as well as the winner (the best model) in the shared task, which integrates bi-directional representations, attention residuals, the label forcing technique and CRF. Experiments indicate that the proposed best model is more effective in capturing global information between the semantic components in the slot filling task than existing models.

This work was supported by FortifyEdge. We would like to specially thank Mr Hyunsuk (David) Chung for his help and guidance for this research work.