Exploring Emotion-Sensitive LLM-Based Conversational AI

Within the context of customer service, emotional labor involves employees managing their feelings and expressions to promote customer satisfaction (Hochschild, 1979). Beyond the actual service provided, customers are more likely to trust an employee or organization when their emotional needs are met, particularly if these emotions are perceived as genuine. This understanding has motivated increased interest in embedding emotional-sensitivity into customer service chatbots (Bilquise et al., 2022; Li et al., 2021; Pamungkas, 2019). However, it is unclear if emotions from an AI would have the same effect. For example, a recent study found that emotion-sensitivity backfired when customers were aware that it was generated by AI, leading them to assume it was inauthentic (Han et al., 2023). Yet the growing sophistication of large language models (LLMs) suggests that these systems can convey emotionally nuanced language that might be perceived as authentic.

To examine this, we performed a study using LLMs to simulate an IT customer service interaction, manipulating the emotional sensitivity of the model. Our preliminary findings indicate that while the addition of emotional sensitivity did not alter the problem-solving abilities of the system (two-thirds of participants reported having their issue resolved and this was the same across both conditions), it did significantly increase their trust and belief in the competence of the emotionally-sensitive system. These results align with findings from emotional labor and reinforce attempts to integrate emotion into such systems. However, they also raise concerns about users potentially over-trusting systems that exhibit human-like emotional expressions (Hancock et al., 2020). Future work will need to investigate if user trust is appropriately calibrated when interacting with emotionally-sensitive customer service agents (e.g., do people under-trust non-emotional agents or over-trust emotional ones?) and examine the longer-term consequences of such technology for both customers and employees.

Empathy has long been recognized as a key factor in enhancing customer satisfaction within the service industry (Kernbach and Schutte, 2005). Well-tailored emotional responses from a customer service agent may help regulate the emotion of an impassioned customer (Delcourt et al., 2013). Early evidence suggests that chatbots may serve to benefit from increased emotional sensitivity in the same way that humans do, although this relationship has not yet been studied comprehensively (Gelbrich et al., 2021). As a novel technology, AI chatbot systems must also build trustworthiness to increase user satisfaction and perceived competence (Huang et al., 2024).

Given this context, it is important to understand whether emotional sensitivity in chatbots enhances users’ perceptions of competence, supportiveness, and overall pleasantness in customer service interactions. Our study aims to explore the relationship between emotional sensitivity and quality of customer service. To access emotional sensitivity, we employ sentiment analysis techniques such as VADER and use the current state-of-the-art LLM chatbots to simulate emotionally nuanced interactions (Hutto and Gilbert, 2014; Mehmood and Balakrishnan, 2020).

Our analysis indicated that both chatbots have equivalent problem-solving performance, with two-thirds of participants reporting their problem solved in both cases. The analysis of emotional states using the PANAS scale revealed a significant decrease in negative emotions following interactions with either chatbot. However, the two chatbots performance began to deviate in terms of user perceptions and experiences. Participants who interacted with the emotionally sensitive chatbot scored the bot higher in terms of capability, knowledge, trustworthiness, supportiveness, understanding, and willingness to use it in the future when compared to participants who interacted with the emotionally insensitive bot. While both chatbots were equally effective in resolving users’ concerns, participants in the emotional group tended to perceive their interactions in more socially and emotionally engaging terms, which aligns with theories that imbuing technology with emotional sensitivity can enhance user acceptance and satisfaction by creating more human-like interactions (Rapp et al., 2021).

Our results demonstrated statistically significant findings regarding our second hypothesis, that users would rate the emotion-sensitive chatbot at a higher competence level than the emotion insensitive chatbot. However, our experiment revealed no significant differences in regards to our first hypothesis, suggesting that the two chatbots did not impact the positivity of the user’s emotions differently. Thus, while users did not mirror the bot’s emotional state, they did tend to associate emotional sensitivity with competence. In the customer service field, these findings demonstrate that emotional receptiveness of a customer service agent is an important component of increasing satisfaction, especially in the often high-pressure setting of customer service support.

There are several points to note regarding the validity and future directions of this study. Firstly, the sample size of the study was extraordinarily small; larger studies would be encouraged to improve the validity of results both in terms of perceived chatbot competence and emotional impact. The participant pool was also limited to relatives and close friends, which all might have similar interactions with customer support services and AI chatbots in general. We hypothesize that since we prompted participants with fictitious scenarios (that our team generated), this could have had an effect on the participants’ interaction with the chatbot and their perceived emotions.

Additionally, our experimental design, which introduced fictitious scenarios, may have influenced participants’ interactions and emotional responses towards the chatbots. Future studies could explore alternative scenario designs to minimize potential biases and improve consistency. Experimenting with different prompts could further elucidate their impact on user receptiveness.

Methodologically, our emotional sentiment analysis was based on single user messages, limiting the contextual understanding of emotional dynamics. In future experiments, sentiment analysis can be run using the context of entire conversations to better understand the impact that emotional context has on user emotion. We also acknowledge the limitations associated with our use of VADER for sentiment analysis. While VADER was efficient for general sentiment analysis, its rule-based nature may have overlooked nuanced emotional expressions crucial to our study. Given the topic’s alignment with affective computing, the reliance on VADER could be perceived as a straightforward approach that ensures objectivity, but might not delve deeply into the subtleties of affective states. Future research could benefit from more sophisticated models like BERT-based approaches (Devlin, 2018) to account for the contextual and semantic intricacies of the data, and therefore capture subtler emotional nuances effectively.

By conducting a between-subjects study to compare user sentiments throughout interactions with an emotion-sensitive chatbot and an emotion-insensitive chatbot, we were able to find that perceptions of chatbot trustworthiness and competence were higher in the case of the emotion-sensitive chatbot. Still, we discovered that there was no significant difference between emotional states following interactions with either chatbot, suggesting that emotional sensitivity may prove to be more useful for competence ratings than emotional management in an IT context. In future studies, the level of this impact may be examined by controlling for variables such as prompts, IT scenarios, conversation context, and more. Future studies may also be able to better contextualize these results with experiments that integrate real-life IT scenarios rather than simulated ones. With the discovery that emotional sensitivity may be perceived to make a digital chatbot system more competent, customer service support systems may be able to improve user satisfaction.

This pilot study was performed as part of a class project in an Affective Computing course and is intended to lay the groundwork for more rigorous future evaluations. The experimental design conforms to a protocol that was classified as exempt by our university’s ethical board (i.e., involves minimal risks). Participants were a convenience sample of friends and family members who were well-acquainted with the research but blind in advance to the nature of the study. As such, the diversity of the sample was limited, and our sample may not be representative of typical users of customer support technology due to biases. Thus, readers should take caution when generalizing the implications of these findings to real-world contexts.

Finally, we acknowledge that the use of emotional AI in customer service applications could be viewed as manipulative. Real-world applications of this technology should consult ACM’s Code of Ethics and Professional Conduct and be sensitive to user autonomy. Users should be allowed to opt into the use of emotional behaviors. Future research should carefully consider the ethical implications of this technology in for-profit applications.