Leveraging the power of transformers for guilt detection in text

Guilt is an emotion characterized by the feeling of unease or remorse due to perceived moral transgressions or failure to meet personal standards. According to Rawlings (1970) guilt is particularly complex due to its different manifestations, which can be categorized as reactive, existential, or anticipated guilt, depending on whether it pertains to past, present, or future actions, respectively. And although feeling guilty serves as a moral compass, promoting self-reflection, empathy, personal growth, and positive change, when experienced in overabundance, culpability can have adverse effects on one’s emotional well-being including stress, anxiety, and depression Tangney and Dearing (2002). Thus, the ability to detect guilt becomes crucial as it paves the way for future endeavors aimed at averting negative outcomes.

From the perspective of Natural Language Processing (NLP), the detection of emotions such as guilt has gained significant popularity in recent years with a notable increment in emotions to be detected, the source of the annotation and the modalities being used Alvarez-Gonzalez et al. (2021); Garcia-Garcia et al. (2017). Certainly, the emergence of advanced deep learning techniques like language models Kusal et al. (2023), has facilitated the examination of emotions from a linguistic standpoint, as these models can efficiently process vast amounts of textual data and interpret emotions accurately Acheampong et al. (2020). However, despite the substantial interest in emotion detection as a whole, there is still a need for further research on specific emotions. Guilt, in particular, has been relatively overshadowed by more popular and general emotions such as sadness or happiness Bolo et al. (2023); Kim and Hansen (2010). And to the best of our knowledge, there is only one existing work Meque et al. (2023) that explores guilt, with classic machine learning. This study aims to explore the capability of language models in guilt detection. We also introduce GuiltBERT, a masked language model built upon BERT Devlin et al. (2018), as an emotion-specific approach to identifying guilt-related linguistic cues and patterns. This work makes significant contributions by shedding light on the detection of the less explored emotion of guilt and enabling further research in this domain. Our model achieves comparable results to existing emotion detection models while setting a new state-of-the-art for guilt detection.

The rest of the paper is organized as follows: Section 2 offers a succinct overview of prior research on guilt detection and the different datasets that include guilt as a target emotion for detection. Section 3 examines existing datasets for the study of text-based emotion detection, specifically those that include guilt. Section 4 provides a detailed description of the GuilBERT architecture its training process and the state-of-the-art models used for the evaluation. Section 5 presents the evaluation results of GuilBERT. Section 6 presents a comprehensive analysis of the strengths and limitations of our model regarding the detection of guilt and other emotions. Lastly, in section 7 we conclude by highlighting our contributions and outlining avenues for future research.

Guilt has predominantly been studied within the humanities. Particularly, the literature shows the detection of guilt has been vastly motivated in forensic science because of its applications in criminal investigation and legal proceedings. However, these works focus on detecting deception and determining if a person "is" guilty of a certain prohibited act, rather than examining if the subject "feels" guilty. Our work is centered on the emotional side instead, in which individuals feel responsible for wrongdoing or violating their own ethical standards because of one’s actions or omissions. In this sense, the detection of guilt is closer to the detection of remorse, self-blame, and in some cases, regret and suicide ideation. There are few works on this side, some approach the detection using body language expressions Julle-Danière et al. (2020) or audio recordings Shinde et al. (2020) as input.

Regarding text-based approaches, the ISEAR dataset Scherer and Wallbott (1994) was probably the first to include guilt as one of the classic four emotions (joy, anger, fear, and sadness), making the guilt emotion more visible and providing researchers with a valuable benchmark to evaluate guilt detection models. Guilt was included because is considered to be essentially limited to the human species, like other self-reflexive emotions such as disgust and shame. In this work, the author also found that guilt is quite similar to shame except for longer duration and the absence of marked physiological response symptoms.

Unlike more popular emotions such as happiness and anger Kim and Hansen (2010); Bolo et al. (2023), the emotion of guilt has been studied only in a multiclass setting rather than individually, presumably because of its complexity, low resource availability, and minor commercial applications. The next subsection presents the publicly available datasets used for training and evaluating guilt detection models, contributing to the advancement of research in this area.

In this section, we present our experimentation setting and the procedure we followed to process the datasets being used for the tasks of general emotion detection and guilt detection.

In this section, we present the outcomes of our evaluations on three distinct datasets, showcasing the results of our proposed models for general emotion detection and guilt detection. Table 5 and Table 6 illustrate the results obtained for the emotion detection task using the ISEAR and the VIC balanced datasets respectively, while Table 7 presents the results obtained for the guilt detection task using the Vic binary dataset. To assess the performance of our models and provide meaningful comparisons we report the standard metrics of precision, recall, and F1 scores.

The results of our experiments for emotion detection showcase RoBERTa slightly outperforms the BERT and GuiltBERT models. Nonetheless, we remark the GuiltBERT outperforms the BERT and RoBERTa models in the guilt detection task

To provide deeper insights into its performance, we present three confusion matrices showcasing GuiltBERT’s classification results for each dataset. By analyzing these visual representations, we gain a comprehensive understanding of the GuiltBERT model performance in emotion classification, particularly in the context of guilt detection.

The findings reveal significant insights into GuiltBERT’s capabilities for capturing the emotion of guilt. It demonstrates a remarkable accuracy in correctly identifying guilt, as evidenced by consistently high numbers of true positives across all experiments. This suggests that GuiltBERT excels in accurately detecting and classifying instances of guilt within the given datasets.

However, when it comes to emotion detection more broadly, GuiltBERT tends to occasionally confuse guilt with the emotions of disgust and shame. This can be attributed to the close relationship and overlapping characteristics between these negative emotions. While GuiltBERT’s performance remains impressive, these instances of confusion highlight the complexity and intricacy of distinguishing between closely related emotions.

Researchers are developing increasingly sophisticated algorithms and models to identify, classify, and analyze emotional experiences in natural language text, and the study of emotions in NLP is a rapidly expanding field. However, despite the increasing interest in applying natural language processing techniques for emotion identification on social media, the emotion of guilt has received limited attention.

In this work, we explored the capability of language models in guilt detection. We also introduce GuiltBERT, a masked language model built upon BERT Devlin et al. (2018), as an emotion-specific approach to identifying guilt-related linguistic cues and patterns. This work makes significant contributions by shedding light on the detection of the less explored emotion of guilt and enabling further research in this domain. Our model achieves comparable results to existing emotion detection models while setting a new state-of-the-art for guilt detection.

As for future work, we plan to upgrade our GuiltBERT to be finetuned using Roberta and other architectures. Additionally, is worth exploring a new task called "explainable guilt" consisting of extracting the fragment of text implying a guilt emotion.

The work was done with partial support from the Mexican Government through the grant A1-S-47854 of the CONACYT, Mexico, grants 20211784, 20211884, and 20211178 of the Secretaría de Investigación y Posgrado of the Instituto Politécnico Nacional, Mexico. The authors thank the CONACYT for the computing resources brought to them through the Plataforma de Aprendizaje Profundo para Tecnologías del Lenguaje of the Laboratorio de Supercómputo of the INAOE, Mexico, and acknowledge the support of Microsoft through the Microsoft Latin America Ph.D. Award.