Why Do We Laugh? Annotation and Taxonomy Generation
for Laughable Contexts in Spontaneous Text Conversation

In human dialogue, laughter serves as a communicative signal conveying humor, empathy, surprise, or social bonding Norrick (1993); Glenn (2003); Attardo (2009). However, its mechanisms are complex and multifaceted, and understanding them remains a long-term challenge for dialogue systems aiming to achieve human-like interaction Tian et al. (2016); Türker et al. (2017); Inoue et al. (2022); Ludusan and Wagner (2023); Perkins Booker et al. (2024). Furthermore, traditional approaches to modeling laughter and humor have often been limited to scenarios involving explicit auditory or visual stimuli, with few addressing the subtle contextual nuances present in spontaneous dialogue Bertero and Fung (2016); Choube and Soleymani (2020); Jentzsch and Kersting (2023); Ko et al. (2023); Hessel et al. (2023). Therefore, elucidating the underlying reasons for laughter in spontaneous dialogue data can contribute to making large language model (LLM)-based dialogue more natural and empathetic. However, annotating the reasons for laughter in any formalized manner has been prohibitively time- and labor-intensive, leaving the field largely reliant on qualitative approaches through conversational analysis.

In this study, we address the question of “why do we laugh?” from an informatics perspective by proposing a semi-automated approach to constructing taxonomy labels for the reasons of laughter. First, to identify target segments, multiple annotators were asked to perform a simple binary classification on each utterance in a dialogue data, determining whether it was “laughable” or not, as shown in Table 1. Subsequently, for contexts labeled as “laughable” based on the majority voting, we used an LLM (GPT-4o) to generate the reasoning sentence behind this judgment and further classified these generated reasons into distinct categories (taxonomy labels). This semi-automated taxonomy generation approach is generalizable and can be particularly effective in scenarios where manual annotation is limited to simpler labels, such as emotion labeling.

The purpose of this research is to contribute toward more nuanced conversational AI systems that can recognize and even anticipate moments for laughter, ultimately fostering more natural interactions between humans and machines. Ideally, such systems should be able to respond with the correct acoustics, delay, and consider group size for different laughter types Truong and Trouvain (2012). Our findings reveal that AI can improve our understanding of laughter and offer a foundation for future research in AI context-sensitive recognition.

We annotated laughable contexts in the RealPersonaChat dataset Yamashita et al. (2023). This textual data contains one-on-one Japanese spontaneous conversation where participants chat without assuming assigned personas. It includes approximately 30 utterances per conversation, totaling around 14,000 dialogues. We annotated 900 dialogues, with plans to annotate the remainder in future work.

During the annotation process, each annotator reviewed each dialogue and, after the initial two greeting utterances, made a binary decision for whether the next person would laugh (laughable) or not. Five annotators assigned these binary labels to each utterance. Table 2 summarizes the agreement amongst annotators for laughable labels, which we refer to as “laughable agreement”. While some samples showed clear agreement (either all or none of the annotators marked them as laughable), there were also numerous split samples, highlighting the subjectivity and complexity of the task. If we applied a majority voting process, 3,739 contexts (14.8%) were labeled as laughable, and 21,550 contexts (85.2%) as non-laughable.

Table 1 illustrates a laughable context example. In this dialogue, person A’s final utterance is self-contradictory, requiring high-level comprehension of the dialogue context. These annotations underscore the significance of cultural context and conversational flow in interpreting laughter cues.

Towards developing human-like laughter behaviors in LLMs, we investigated the reasons behind human annotators’ recognition of laughable contexts. In this section, we used only samples with majority labels marked as laughable (3,739 samples). Since manual annotations are costly, we utilized GPT-4o to generate explanations for the human judgments. For example, a generated reason for the example context in Table 1 is: {quoting} If we were to speculate on the reasons a third party might judge that Person A laughed at Person B’s final remark in this conversation, the following points can be considered:

Element of Humor: Person B’s comment, “My husband doesn’t seem to listen to me. Huh, that’s strange.,” contains a touch of self-deprecating humor. This lighthearted tone, making fun of their own situation, can be amusing to the listener. (…)

Relaxed Atmosphere: The overall tone of the conversation seems light and relaxed, and Person B’s comment might have been perceived as a playful joke in line with this mood.

A combination of these factors may have led the third party to interpret that Person A laughed in response to Person B’s remark.

Next, we attempted to summarize the generated reasoning text for laughable contexts by following a taxonomy generation approach using LLMs Wan et al. (2024). First, we randomly divided the generated reasons into smaller portions, each representing 5% of the data. Starting with the initial one portion, we used GPT-4o again to generate taxonomy labels and explanations, manually validating them as needed. We then iteratively updated the taxonomy by having the LLM re-generate the taxonomy information based on the previous taxonomy information and the portion data, until all data was processed. This resulted in the generation of ten taxonomy labels, summarized in Table 3, including categories like (1) Empathy and Affinity and (2) Humor and Surprise.

After generating the taxonomy labels, we again used the LLM to assign labels to each reason sample, allowing for multiple labels per sample. The results of this labeling are summarized on the right side of Table 3. While dominant categories like (1) Empathy and Affinity were observed, a substantial number of samples were also labeled under other categories such as (4) Self-Disclosure and Friendliness and (5) Cultural Background and Shared Understanding. This varied distribution supports the validity of the generated taxonomy labels. A correlation matrix for the taxonomy labels is reported in Appendix A.

Finally, we investigated related conversational analysis studies, as listed on the right side of Table 3. These findings further support the explanatory power of our generated taxonomy labels within the framework of conversational analysis research.

We then examined how much LLMs, specifically GPT-4o, can recognize the laughable contexts in spontaneous text conversation. The model was tested in a zero-shot setting, instructed to first analyze the conversational context and then determine its laughability as a binary. The provided prompt included a task description for laughable context recognition, followed by a Chain-of-Thought (CoT) reasoning approach to encourage the model to consider the reasoning behind its decision step by step. We evaluated GPT-4o’s performance against the majority labels, achieving an F1 score of 43.14%, with a precision of 41.66% and recall of 44.72%. While this score was significantly above the chance level (14.8%), capturing the nuanced subtleties of conversational humor remains challenging.

We then further examined the LLM’s performance on each generated taxonomy label. Table 4 shows the distribution of binary outputs by GPT-4o and its accuracy within each label. First, the primary labels, from (1) to (3), showed similar accuracy rates, ranging from 40% to 50%. Additionally, we observed comparatively higher scores for (5) Cultural Background and Shared Understanding, (7) Self-Deprecating Humor, and (8) Defying Expectations, suggesting that the current LLM may effectively capture these contexts. In contrast, categories like (6) Nostalgia and Fondness and (9) Positive Energy displayed lower accuracy, potentially highlighting limitations in the LLM’s understanding.

Table 5 presents an example dialogue context where the LLM marked non-laughable for the final utterance from person A, despite a positive majority label with a (6) Nostalgia and Fondness reason. This context was also assigned the (2) Humor and Surprise and (3) Relaxed Atmosphere labels. In this example, the participants discuss a nostalgic memory of drinking a powdered beverage with milk. The last utterance evokes nostalgia, implicitly inviting laughter. Here, capturing person A’s sentiment seems to be difficult for the current LLM, but is essential for appropriate laughter response.

Table 6 provides an example for (9) Positive Energy label. This context was also assigned the (1) Empathy and Affinity and (2) Humor and Surprise labels. The participants discussed a challenging experience with childcare, but in the final utterance, person A reflects positively on the experience after some time has passed. Although the story itself recounts a difficult time, it is now viewed positively, making it laughable. This example suggests that the LLM needs to comprehend the temporal structure of the story and the person’s current feelings to accurately interpret the context as laughable.

This study investigated laughter in the context of conversational AI by annotating laughable contexts within a Japanese text dialogue dataset. A taxonomy of ten distinct reasons for laughter was generated by an LLM, providing valuable insights into the multifaceted nature of laughter. Subsequently, this study evaluated the ability of GPT-4o to recognize those laughable contexts. While the model’s performance surpassed chance levels, it highlighted the inherent challenges in capturing the nuances of conversational humor.

This automated approach employed for reasoning and taxonomy generation with LLMs can be applied in other scenarios where only binary (or simplified) decision labels from human annotators are available, yet more fine-grained explanations are required. Future work will focus on expanding the dataset to cover other languages and cultural contexts, validating the generated taxonomy by incorporating additional linguistic research perspectives, exploring multimodal approaches, and including spoken dialogue to enhance AI’s understanding of humor and social interaction.

This work was supported by JST PREST JPMJPR24I4 and JSPS KAKENHI JP23K16901.