Beneath the Surface: Unveiling Harmful Memes with Multimodal Reasoning Distilled from Large Language Models

Harmful meme detection is a rapidly growing area in the research community, driven by the recent availability of large meme benchmarks Kiela et al. (2019); Suryawanshi et al. (2020); Pramanick et al. (2021a). The Hateful Memes Challenge organized by Facebook Kiela et al. (2020) further encouraged researchers to develop solutions for detecting harmful memes in hate speech Das et al. (2020). More recently, Pramanick et al. (2021a) firstly defined the harmful meme concept and demonstrated its dependence on contextual factors. The complex nature of memes, which often rely on multiple modalities, makes them challenging and struggle to yield good performance only using unimodal detection methods Simonyan and Zisserman (2014); He et al. (2016); Devlin et al. (2019). Therefore, recent studies in this area attempted to apply multimodal approaches on the harmful meme detection task.

Previous studies have employed classical two-stream models that integrate text and vision features, which are learned from text and image encoders, using attention-based mechanisms and multimodal fusion techniques for classifying harmful memes Kiela et al. (2019, 2020); Suryawanshi et al. (2020). Another branch was to fine-tune pre-trained multimodal models specifically for the task Lippe et al. (2020); Muennighoff (2020); Velioglu and Rose (2020); Hee et al. (2022). Recent related efforts have also sought to explore the use of data augmentation techniques Zhou et al. (2021); Zhu et al. (2022), ensemble methods Zhu (2020); Velioglu and Rose (2020); Sandulescu (2020) and harmful target disentanglement Lee et al. (2021). More recently, Pramanick et al. (2021b) proposed a multimodal framework by using global and local perspectives to detect harmful memes which achieves state-of-the-art performances. A follow-up prompt-based approach Cao et al. (2022) attempted to concatenate the meme text and extracted image captions to fine-tune masked language models Liu et al. (2019) for harmful meme detection. However, existing solutions only capture the superficial signals of different modalities in memes in an end-to-end manner, which largely ignore explicit deductive reasoning to guide the model for understanding background knowledge about the complex and diverse relations between the visual and textual elements.

Recently, LLMs have demonstrated remarkable capability in complex reasoning Brown et al. (2020); Thoppilan et al. (2022); Rae et al. (2021); Chowdhery et al. (2022), such as generating intermediate inference procedures before the final output Nye et al. (2021); Wei et al. (2022); Kojima et al. (2022); Zhang et al. (2022). Unfortunately, the large size of LLMs restricts their deployment on detecting harmful memes with different modalities, regardless of how they are enhanced with strategetic text prompting. Knowledge distillation has been successfully used to transfer knowledge from larger, more competent teacher models into smaller student models affordable for practical applications Buciluǎ et al. (2006); Hinton et al. (2015); Beyer et al. (2022). However, existing researches on knowledge distillation from LLMs Wang et al. (2022); Ho et al. (2022); Magister et al. (2022) only consider the language modality, they are not suitable for harmful meme detection because harmful memes can convey holistic synergy information through multimodal features. In this work, we conduct abductive reasoning with LLMs, which further advocates a multimodal reasoning paradigm to fine-tune smaller language models (LMs) for harmful meme detection.

In this paper, we propose to utilize abductive reasoning with multimodal inputs to train smaller downstream models. LLMs can produce natural language rationales unveiling the implicit meaning beneath the surface of the memes to justify the reason why the meme is harmful or not. This shares a similar intuition as heuristic teaching Pintrich and Schunk (2002) where a teacher who has rich experience and knowledge can impart to students the correct way of thinking and reasoning based on questions with corresponding answers. The students then learn how to deduce their own ways to the correct answers from questions accordingly. Thus we aim to activate explicit reasoning knowledge in LLMs as a teacher model, e.g., contextual and cultural information related to memes, to guide our model to strengthen harmfulness prediction.

Given a meme sample $M=\{y,\mathcal{I},\mathcal{T}\}$ from the training data, to prompt large language models in uniform language modality, we first extract the text caption $\tilde{\mathcal{I}}$ of the image $\mathcal{I}$ by off-the-shelf captioning models Mokady et al. (2021). Then we curate a template $p$ that consists of a triplet $\{y,\tilde{\mathcal{I}},\mathcal{T}\}$ as observed attributes, to prompt the LLMs to generate a rationale $r$ that elicits the reasoning knowledge about how to infer the harmfulness label $y$ based on the interplay of the meme text $\mathcal{T}$ and the image caption $\tilde{\mathcal{I}}$ as illustrated in Figure 2. Specifically, we design $p$ as:

“Given a Text: [$\mathcal{T}$], which is embedded in an Image: [$\tilde{\mathcal{I}}$]; and a harmfulness label [$y$], please give me a streamlined rationale associated with the meme, without explicitly indicating the label, for how it is reasoned as [$y$].”

As we clarify the ground-truth harmfulness label in the observed attributes of the prompt, the hallucination issue Bang et al. (2023) of LLMs could be effectively alleviated. Because the rich contextual background knowledge could be activated by abductive reasoning based on the ground truth and invalid rationales are naturally filtered out.

Since we utilize image captions to represent the meme images, we could perform abductive reasoning with large language models pre-trained with language modality. However, only using the captions as opposed to original vision features may suffer from a lack of mutual synergy in the representation space of different modalities in memes due to the inductive bias of possible information loss in the captioning process. On the other hand, LLMs can be used to conduct abductive reasoning only for the training data whose harmfulness label is given in prior but is challenging to be fine-tuned for this task due to the huge amount of model parameters. To facilitate the interactions between the meme text and the image, we propose to fine-tune a smaller language model for the harmful meme detection task, which allows flexibility in adjusting model architectures to incorporate multimodal features and is more lightweight for task-specific fine-tuning.

In this section, we train a small language model as a student model distilled from the LLMs with multimodal reasoning knowledge. Specifically, we leverage generated rationales from LLMs as informative supervision, to fine-tune a smaller pre-trained language model to excavate the rich inter-relationship between language and vision modalities of memes.

During the first fine-tuning stage, we conducted explicit deductive reasoning to empower our model with the capability of multimodal reasoning distilled from LLMs. As the goal of this task is to determine whether the meme is harmful or not, we conduct the second fine-tuning stage for Harmfulness Inference, which shares the same model architecture, parameters, and encoding procedure as Sec. 3.2 but differs in the decoding output. To make the output consistent with harmfulness prediction, the smaller model $f$ is further trained by minimizing the following inference loss:

where the cross-entropy loss is computed between the generated text and ground-truth harmfulness label $y$. With the generative objective Raffel et al. (2020) adapted to the previous Reasoning Distillation stage, the prior reasoning knowledge absorbed in Reasoning Distillation could be well induced for Harmfulness Inference.

Table 1 shows the performance of our proposed method versus all the compared methods on the Harm-C, Harm-P and FHM datasets. It is observed that 1) The performance of the baselines in the first group is obviously poor due to only unimodal features like text-only or image-only being captured. In comparison, the other baselines exploit the multimodal features from both the text and image in memes. 2) The multimodal models in the second group outperform the unimodal ones. The early-fusion models with multimodal pre-training (i.e., VisualBERT COCO and ViLBERT CC) outperform that of the simple fusion with unimodal pre-training (i.e., Late Fusion and MMBT) on Harm-C/P datasets, while MOMENTA performs best in the second group by considering global and local information of memes. 3) However, as the images in FHM dataset are more informative and high-quality, MaskPrompt yields the best performance among all the baselines by incorporating additional extracted entities and demographic information of the image into the masked language models, besides just captioning the image into the prompt.

Our proposed Mr.Harm improves over the best baselines by 2.63%, 1.31%, and 9.86% in terms of Macro-F1 score on Harm-C, Harm-P, and FHM datasets, respectively. We observe that 1) the improvement on the Harm-P dataset is relatively milder than that on the other two datasets. Meanwhile, all the baselines just have tiny differences among their performances on Harm-P. We speculate the reason falls into the smaller dataset scale of Harm-P which only contains politics-related harmful memes. 2) A similar trend can also be observed in Harm-C and FHM datasets: the more challenging the dataset is, the greater performance improvement Mr.Harm achieves. Our model performs flexibly and stably across all datasets with its keen judgment on harmful memes. This is because all the baselines are only designed at the recognition level, but Mr.Harm is further empowered with multimodal reasoning knowledge distilled from LLMs to unearth harmful content from the seemly uncorrelated text and image modalities of memes.

We perform ablative studies on several variants of Mr.Harm: 1) w/o Reasoning Distillation: Simply fine-tune the smaller language models in the stage of Harmfulness Inference without the stage of Reasoning Distillation based on LLMs; 2) w/o Visual Features: Discard the features from the meme image while keeping those from the meme text; 3) w/o Multimodal Fusion: Instead of the fusion mechanism on the multimodal features in our language model, we only append the lingual features from image captioning together with the meme text during encoding; 4) w/o Two-stage Training: Concatenate the rationales generated from LLMs and golden harmfulness label as the target for model training, to replace the two-stage training paradigm; 5) w/o Fine-tuning Small LMs: Directly prompt the representative large language model ChatGPT based on InstructGPT Ouyang et al. (2022) for harmful meme detection.

As demonstrated in Table 2, the ablative models suffer different degrees of performance degradation, indicating the effectiveness of our proposed components for harmful meme detection with multimodal reasoning distilled from LLMs. Specifically, the performance of Mr.Harm significantly decreases in the ‘w/o Reasoning Distillation’ setting due to the lack of multimodal reasoning knowledge transferred from LLMs about the seemly uncorrelated modalities in memes. The ‘w/o Visual Features’ setting also achieves worse performance than Mr.Harm, suggesting that the visual representations are complementary to the meme text for harm-indicative pattern extraction in the language model. Mr.Harm makes improvements over ‘w/o Multimodal Fusion’, which implies the promoting role of our fusion mechanism that incorporates original vision features into the language model, hardly compromised when there could be severe information loss in the captioning process. Moreover, the ‘w/o Two-stage Training’ setting leads to large-margin performance degradation, which verifies the effectiveness of our two-stage training paradigm. This is because this setting causes mutual interference between intermediate reasoning and final prediction, which affects the convergence effect of harmfulness inference and damages the model’s performance and stability. Compared with Mr.Harm, the performance of ‘w/o Fine-tuning Small LMs’ also significantly decreases, highlighting the importance of abductive reasoning with LLMs to alleviate the hallucination issue during deductive reasoning for harmfulness prediction.

Note that our smaller language model is explicitly trained in the Reasoning Distillation stage for rationale generation to distill multimodal reasoning knowledge from LLMs. Although intermediate reasoning is not the final target output for harmful meme detection, after the first fine-tuning stage, we elicit reasonable thoughts from our smaller language model with the test samples as input, to understand the cognition view of our proposed Mr.Harm on the test meme samples more transparently and intuitively, as exemplified in Figure 3.

From the visualized intermediate reasoning, we observe that 1) our model could understand the multimodal information related to the meme text (in green) and image (in blue) with commonsense knowledge. For example, in Figure 3(a), the recognized “politician” in the image could be related to “president trump”, which could be linked to the “AMERICA” in the text; in Figure 3(b), the recognized “flag” in the image could be cognized to satire “the democrat party” in the text; and in terms of Figure 3(c), the “goat humper” and “pigs” in the text could be associated with the attacks to “a Muslim man” recognized in the image. 2) Furthermore, our model learns to cognize the interplay (in pink) of multimodal information with advanced reasoning. Benefitting from the rich multimodal understanding of the memes, the perpetuates harmful stereotypes could be reasoned over the context to the target like “who affected by the pandemic” in Figure 3(a), “the democrat party” in Figure 3(b), and “the Muslim” in Figure 3(c). In this way, the rich correlation beneath the surface of the meme text and image could be excavated to facilitate harmfulness inference with better reasoning knowledge by harnessing advanced LLMs. Such readable pieces of rationales are also potentially valuable for aiding human checkers to verify the final answer predicted by our model.

To better understand the behavior of our model and facilitate future studies, we conduct an error analysis on the wrongly predicted memes by our proposed framework. We found that the major error exists in that our framework still cannot fully recognize the images that require rich background knowledge though we exploited the advanced cross-attention mechanism by incorporating visual features into the language model. Figure 4 shows two examples of memes wrongly classified by Mr.Harm. For the harmful meme in Figure 4(a), the phrase “and for my next class project!” suggests that the image is being used for an academic or educational purpose, which can be seen as glorifying or normalizing the behavior depicted in the image. The image features “a group of Ku Klux Klan members walking on a beach”, which is a symbol of white supremacy and racism. The combination of the phrase in the text and the use of imagery associated with hate groups can contribute to the glorification of harmful behaviors and the perpetuation of negative stereotypes, which makes the meme harmful. However, due to the lack of related background knowledge about the Ku Klux Klan members and their wear, our framework cannot well recognize the image correctly during the original vision feature extraction, which leads to error propagation for wrongly concluding that the meme is harmless. Also, in terms of the harmless meme in Figure 4(b), the image that “Jimmy Carter with a smile on his face” is mistakenly recognized as “an older man with a funny expression on his face”, furthermore, the model hallucinates that the meme text “can be considered harmful and offensive to individuals who identify with the politician”, resulting in the wrong prediction that the meme is harmful. Therefore, it is possible to improve Mr.Harm by incorporating more informative vision features and improving language-vision interaction to be capable of understanding the images with more complex background knowledge.

As our two-stage training paradigm requires distilling the reasoning knowledge and leveraging original vision features, we utilize the T5 encoder-decoder architecture Raffel et al. (2020); Chung et al. (2022) to initialize our generative framework. To test the generality of the benefits of our approach to different versions of the backbone, we alter the underlying LMs to other variants in different sizes. As shown in Table 3, one interesting phenomenon is that our model has already achieved outstanding performance on the three benchmarks with the Small (about 60M parameters) or Base ((about 220M parameters)) version as the backbone, which has a smaller size than the state-of-the-art baseline MaskPrompt (over 300M parameters). The Large version of our backbone generally achieved better performance than the other two backbone versions because the larger the fine-tuned LMs, the more it alleviates the hallucination issue Ji et al. (2023). Overall, the results show that our framework does not rely excessively on the size of the backbone to improve performance and is generally effective with different versions of the backbone model.