Emojis as Anchors to Detect
Arabic Offensive Language and Hate Speech

We extracted common emojis that appear mostly in offensive communications from shared datasets in Zampieri et al., (2020) and Chowdhury et al., 2020b , and obtained their emoji variations from https://emojipedia.org/. These emojis include some animals and symbols used for dehumanization and expressing disrespect, anger or disgust. The complete list of emojis used for the data collection and their categories is given in Figure 2.

From a collection of 4.4M Arabic tweets between June 2016 and November 2017, we extracted all tweets having any of the aforementioned emojis. After removing duplicates, near duplicates and very short tweets, we ended up with 12,698 tweets. We show later that tweets collected in another time period (in March 2021) still have high percentages of offensiveness and hate speech.

We created an annotation job on Appen crowdsourcing platform to judge whether a tweet is offensive or not (Job1) and we invited annotators from all Arab countries.³³3We described offensive language to annotators as any rude and socially unaccepted language used by some users to offend a person or a group. This includes also vulgar and swear words. Quality was assured using 200 questions that we manually obtained gold labels for (hidden test questions). Annotators should pass 80% of them to continue.⁴⁴4We paid $15 per hour of work to conform to the minimum wage rate in the US. Each tweet was judged by 3 annotators and more than 190 annotators contributed to this job. Such a large number is needed for a subjective task like judging tweet offensiveness. Inter-Annotator Agreement (IAA) using Cohen’s kappa ($\kappa$) value was 0.82, which indicates high quality annotations (Landis and Koch, (1977)).

Annotators fully agree (3 out of 3) in 65% of the tweets. For better quality, we hired and trained an expert linguist who is familiar with different dialects to carefully check all tweets having different opinions. This resulted in changing 18% of the labels (w.r.t to the previously assigned majority label) for the tweets.

We created another job on Appen with all the annotated offensive tweets from Job1 and asked annotators to detect the presence of hate speech. We defined hate speech as any content that contains offensive language targeting group of people based on common characteristics such as: race/ethnicity/nationality, religion/belief, ideology, disability/disease, social class, and gender. We used job settings similar to Job1. Additionally, we asked annotators to judge whether the offensive tweets have profanity or vulgar words, and whether they promote violence. Basic statistics and examples from different annotations are listed in Table 2.

We report our observations about common emojis used to represent offensiveness in below:

• •

  There are additional emojis, unique to our initial seed list, frequently used in the annotated offensive tweets. This class of emojis (see “New emojis” row in Figure 3) includes instances like spit/drops (65% of tweets having it was labeled as offensive) to hammer (18% offensive). This suggests a potential for enriching the initial emoji seed list and expanding the dataset in an iterative way.

• •

  Top offensive emojis are shown in Figure 3 (mostly animals). If a tweet has any of them, most likely it is labeled as offensive (pig (84% offensive) to middle finger (62%)). Similar emojis appear in hate speech tweets with percentages range from 36% to 18% respectively.

• •

  Top vulgar emojis are mostly used in tweets having adult content (e.g. adult ads).

• •

  For violence category, the most common emojis are: knife, punch and shoe in order.

We collected tweets from a different time period (in March 2021) to study the usage pattern of offensive emojis over time. For this, we selected six emojis from Figure 3 and their corresponding words to extract their tweets.⁵⁵5We ignored ”high heel shoe” emoji as it is used dominantly in adult ads, and ”donkey/horse” emoji as it is ambiguous in its usage (as donkey in some cases and horse in others). “Donkey” emoji is not found in https://emojipedia.org For each emoji and its corresponding word, we then randomly selected 200 tweets and asked an Arabic native speaker to judge the content for offensiveness.⁶⁶6Some words appear less than 200 times in this collection.

Our study suggests that some emojis (see Figure 4) and their corresponding words are widely used in offensive tweets during different periods of time. While middle finger was reported as the strongest offensive emoji in English tweets (Wiegand and Ruppenhofer, (2021)), we found that there are more common offensive emojis appear in our dataset of Arabic tweets such as pig (84%), shoe (77%) and dog (68%) (first in “Top offensive emojis” row in Figure 4). Tweets having middle finger emoji were tagged as offensive in 62% of the cases. We found annotation errors due to short context in some cases, but some users use middle finger mistakenly instead of index finger⁷⁷7We anticipate middle finger emoji is not widely understood as offensive by all Arab users. as shown in Figure 5.

As observed in Donato and Paggio, (2017), emojis are often redundant and convey something already expressed verbally in tweets. Thus, offensive emojis may co-occur with many offensive words. As dialectal Arabic (DA) is widely used on Twitter, it’s very hard to list all variations of offensive dialectal words. Figure 6 shows an example where offensive emojis co-occurred with offensive dialectal words which confirms the aforementioned observation. It also shows how emojis are used to cover morphological variations such as number and gender.

To partially solve spelling mistakes in the collected tweets, we normalized some letters which are commonly used interchangeably by mistake, namely letters \<أإآ،ة،ى> to letters \<ا،ه،ي> in order. We calculated valence score for normalized words in offensive and clean tweets as described in Conover et al., (2011). The score helps determine the distinctiveness of a given word in a specific class in reference to other classes. Given $N(t,off)$ and $N(t,cln)$, which are the frequency of the term $t$ in offensive tweets ($off$) and clean tweets ($cln$) in order, valence score is computed as:

Figure 7 shows top words with the highest valance score in offensive tweets.⁸⁸8Condition: Valence score ¿= 0.8 and frequency ¿= 5. In addition to some animal words (e.g. dog, sheep), the list contains many dialectal words that are widely used in offensive communications such as \<زق، خرا، زباله> (“zq, xrA, zbAlh” - shit, garbage) and some adult ads words such as \<سكس، ديوث> (“sks, dywv” - sex, cuckold). This proves the efficiency of starting with offensive emojis to collect related offensive words without any preference to dialect or genre. All lists of extracted words with their valence scores will be made publicly available.

For intrinsic evaluation, we manually revised these top offensive words, and we found that 71% of them appear dominantly in offensive communications and can be considered as correct offensive words. However, the remaining 29% are not necessarily offensive and can appear in many clean contexts. This includes named-entities such as \<إيران، بوتين، الحوثي، بشار> (“AyrAn, bwtyn, AlHwvy, b$Ar” - Iran, Putin, Houthi, Bashar) in addition to some words that were concentrated by chance in our offensive tweets. Examples of such words are: \<ذل، بلدك، ذكور> (“*l, bldk, *kwr” - humiliation, your country, males).

We used the aforementioned valence score formula to extract common words appear in all hate speech tweets. Then, we manually filtered and grouped them to extract common targets for hate speech. These targets include women, countries, groups and political parties as listed in Table 3.

We observe that vulgar words include genitals, sexual actions and references in addition to words are used in adult ads. They are mainly written in DA and the most common words are shown in Figure 7.

As violence tweets are rare (0.7% in our collection), we manually analyzed them all to extract common patterns. These patterns are summarized in Table 4. First, we define <head> for head and neck, and <body> for body parts including <head> that can be used as objects for many violence verbs, ex: I will stab [him/his neck]. For future work, we plan to extend this list using synonyms and related words from dictionaries/BERT models. Then, we extract tweets having any of these verbs and targets after applying some morphological expansions.

Typical approach to crawl data from social media, specially from Twitter, is heavily dependent upon task-specific keywords. Preparing such keyword-specific anchors needs a significant knowledge on the language usage. However, using emojis to collect the data removes the initial language dependency and inherited bias from the keyword-search space.

In this Section, we study the efficacy of emojis to collect offensive tweets for other languages. We utilize the proposed pipelines for Bengali (a low-resource language)¹⁰¹⁰10Bengali is the lingua franca of Bengalidesh and parts of India. There are 228 million native speakers, making it the fifth most-spoken native language in the world. Source: https://en.wikipedia.org/wiki/$Bengali_{l}anguage$ and analysed the percentage of offensive tweets obtained using the method. Moreover, we also include analysis on English tweets to indicate how offensive communication and emoji usage are still culture-dependent, if not language.

For Bengali, we collected 200 tweets in August 2021 for five emojis (a total of 1K tweets) from the different groups listed in Figure 2. From these 200, we randomly selected 50 tweets per emoji (a total of 250 tweets) and a Bengali native speaker annotated them for offensiveness. We noticed that $\approx 34\%$ of all the annotated tweets are offensive. We observed that tweets with ‘middle finger’ had most frequent ($50\%$ of tweets) number of offensive content, followed by ‘pile of poo’ emoji (32% of tweets). The detailed distribution is in Figure 10 with examples in Figure 11. We plan to use this method to collect and annotate offensive tweets for other low-resource languages such as Hindi,¹¹¹¹11Pilot annotation of Hindi tweets shows similar trends (52% offensive for ‘middle finger’, and 40% offensive for ‘angry face’.) in addition to Bengali. We keep this as a potential future work.

As usage of emojis for offensive content in English is well-studied, we only collected English tweets having the top Arabic offensive emojis (pig and shoe) in April 2021. We collected more than 8,000 tweets per each, and we randomly selected and annotated 200 tweets for each emoji. We noticed that less than 5% of those tweets are offensive. Often, pig emoji appears in contexts about food while shoe emoji appears in tweets talking about fashion and clothes. Examples are shown in Figure 9.

This suggests that while some emojis are widely used in offensive communications in a particular culture, they might be used completely in different ways in other languages or cultures. Thus, it’s important to keep these differences in mind when collecting data using emojis from different cultures.

In order to design the automated classification system, we fine-tuned the state-of-the-art monolingual (Arabic) pretrained transformer architectures – AraBERT (Antoun et al., (2020)) and QARiB (Abdelali et al., 2021a ). Furthermore, we compared the performance of the monolingual transformers with two multilingual BERT: mBERT (Devlin et al., (2019)) and XLM-RoBERTa (Conneau et al., (2019)). As a baseline, we use a SVM (Platt, (1998)) classifier, due to its reputation as an universal learner and for its ability to learn independently of the dimensionality of the feature space and the class distribution of the training dataset.

From the reported results, presented in Table 6 and Table 7, we observe the monolingual models significantly outperforms the multilingual models. This observation is in-aligned with previous studies (Polignano et al., (2019); Chowdhury et al., 2020a ).

Comparing the monolingual models, for offensive classification, we observe that mixed trained model – QARiB, outperforms AraBERT by a margin of 2.3%. Contrary to that, in hate classification, we noticed AraBERT - trained with formal text – performs the best. However, the performance difference is very small ($0.1\%$) and can be attributed to the randomness mentioned in Devlin et al., (2019).

Our proposed method of collecting an offensive language dataset relies on emojis. While this method yields higher percentage of offensive content compared to existing methods, it’s crucial that models trained on our dataset perform well not only on our test set but also on external datasets. Achieving such generalization can help us to assert with confidence that our method captures characteristics of offensive language that are universal, not specific to our dataset only.

In addition, assessing the generalization of the model to other social media platforms such as Facebook and YouTube, can indicate the universal usage of emoji and also effectiveness of harnessing the extralinguistic information present in emojis for offensive task.

The dataset was collected using the Twitter API. For privacy, we anonymised the dataset by removing usernames and other sensitive user identifier. However, in the future, if any concern is raised for a particular content, we will comply to legitimate concerns by removing the affected tweets from the corpus. Furthermore, to keep privacy and datasets integrity, we are providing tweets by text not the corresponding tweet ids.

Any biases found in the dataset are unintentional, and we do not intend to do harm to any group or individual. The bias in our data, for example towards a particular group is unintentional and is a true representation of the Twitter space during the period of our collection and according to our generic proposed method. Emojis used as seeds to create the collection are extracted automatically from publicly available datasets without any bias in our selection. Collection was obtained over a span of 18 months to have a good diversity of tweets written by many authors.

As for the assigned annotation labels, we follow a well-defined schema and available information to perceive final labels for offensiveness, hate speech and vulgarity. The labels are not a reflection of our opinion. Test questions used for quality control were selected to cover different annotation classes proportionally to their distributions in the collected tweets.

Although more than 190 workers have participated in the annotation process (with at least 80% success ratio for 200 test questions), this doesn’t guarantee perfect quality of annotation. As noted in Mubarak and Darwish, (2014), Arabic Twitter is dominated by tweets from Gulf countries (Saudi Arabia, Kuwait, etc.) while Appen (previously CrowdFlower/Figure8) crowdsourcing platform has $\approx 30\%$ of annotators from Egypt (the most Arabic populated country) (Mubarak and Darwish, (2016)). This may lead to misunderstanding (and incorrect labeling) of portion of tweets especially when they are written in dialects that are not understood by annotators. Moreover, almost three quarters of Arab annotators on Appen are males which may lead to hidden biases in data annotation.

We urge the research community to be aware that our dataset can be used to misuse as like any other social media data. If such misuse is noticed, human moderation is encouraged in order to ensure this does not occur.