Affective Conversational Agents: Understanding Expectations and Personal Influences ^††thanks:  ⁺Corresponding author: [email protected].

To explore people’s perspectives on affective empathy in AI conversational agents, we designed a survey with the following sections:

Demographics and background information. We collected basic demographic information about the participants, including their gender, age, job role, education level, and whether English was their first language.

Personality. To examine the role of personality on personal preferences, we included the 10-item Big Five Personality Inventory [47] which decomposes personality into five main components: openness, conscientiousness, extroversion, agreeableness, and neuroticism.

Emotional Experience. To assess various aspects of participants’ emotional experience, we included the 20-item Toronto Alexithymia Scale [48] which provides an overall score for the inability to recognize or describe one’s own emotions (a.k.a., alexythmia) as well as separate scores for the difficulty to describe and identify feelings, as well as the ability for externally oriented thinking. We also included the 10-item Emotion Regulation Questionnaire (ERQ) [49] which provides scores for cognitive reappraisal and expressive suppression. These measures were selected based on their relevance to emotional intelligence, communication, and empathy, which are key components in the context of AI conversational agents.

Prior experience. To help contextualize the findings of this work, we evaluated participants’ familiarity with the technology by asking them to rate the duration and frequency of their interactions with conversational AI agents, as well as describe some of their primary use cases.

Preferences. To capture preferences on affective empathy for AI conversational agents, participants were asked to indicate whether they would like to interact with an AI agent with simulated affective empathy to achieve different goals. Based on the seminal book of Affective Computing [15], we divided digital affective skills into three main components: the ability to perceive emotions from the user, the ability to respond to users’ emotions, and the ability to simulate its own emotions. For each of these abilities, we provided a definition and some examples (see Table I). These dimensions were chosen to cover some of the core aspects of affective empathy [15], ensuring that our measures provided a nuanced understanding of participants’ preferences in AI conversational agents. For each of these abilities, participants were then asked to rate whether they would like to see them when interacting with an AI agent across 32 application areas.

We selected the list of application areas based on popular applications of text-based chatbots (e.g., browsing and retrieving content, generating personalized stories and narratives), as well as prevalent application areas of emotion recognition [50] (e.g., providing empathetic emotional support and assistance, facilitating customer service interactions). These applications can be grouped into 10 main thematic categories (e.g., customer service and sales, creativity, and content generation) as shown on Figure 1 (top).

Finally, to better understand potential individual variations in terms of affective preferences across all applications, we defined a Preference for Affective Conversations score (PAC) for each participant. We computed this score as follows:

where $n$ is the total number of applications, and $I_{ip}$ is an indicator variable that takes the value 1 if participant $p$ indicated at least one empathy preference for the $i^{th}$ application, and 0 otherwise.

Open-ended scenarios. Since the curated list of application areas might not have captured all possibilities, we asked participants to identify the ideal and worst application scenarios for an AI agent with affective empathy. For the ideal scenario, participants also provided information about the signals they would be willing to share with the AI conversational agent, as well as their preferences regarding the platform and potential channel of communication. To analyze open-ended questions, we conducted a thematic analysis regarding potential areas of application and concerns about affective AI agents. Each survey response was manually coded according to the themes we discovered to facilitate quantitative analysis.

A link to the survey was sent via email to a random set of information workers of a large technology company in the USA to request their preferences in terms of affective AI agents. Participation was voluntary without any form of compensation, and completing the survey took approximately 15 minutes. The study was pre-approved by the Institutional Review Board.

A total of 745 information workers took part in the user study, representing diverse job roles, education levels, age groups, and gender identities. The majority of participants were in the software development/engineering field (55.84%), followed by product management (15.57%), and data science/analytics (3.62%). Other job roles accounted for 21.36% of the participants. The participants’ education levels were predominantly bachelor’s degrees (46.44%) and master’s degrees (35.30%). A smaller percentage held other education levels, with a few preferring not to disclose their education (1.34%). In terms of age distribution, the largest group of participants fell within the 26-35 age range (32.89%), followed by those aged 36-45 (28.72%) and 46-55 (20.81%). A smaller percentage belonged to other age groups (1.74%). Gender-wise, the majority of participants identified as men (70.20%), while 25.64% identified as women, and the remaining 1.47% identified as non-binary/gender diverse, self-described, or preferred not to disclose their gender.

Prior experience with agents can play a role in how people perceive and understand interactions with agents. In our survey, 56.24% of the participants reported having less than 6 months of experience with AI conversational agents, followed by 13.56% with 6 months to 1 year, 8.86% with 1-2 years, and 9.13% had more than 5 years of experience. Regarding the frequency of interaction, 6.58% of participants had never interacted with AI conversational agents, 15.57% had interacted rarely (once or twice in their life), 37.45% interacted occasionally (a few times a year), 22.68% interacted frequently (several times per month), and 17.72% interacted very frequently (multiple times per week or more). This highlights that most of the participants had at least occasional interactions with AI conversational agents.

Analysis of the free-form question regarding frequently explored applications revealed a variety of settings where respondents have used conversational AI agents, including customer service, personal assistance, healthcare, education, finance, e-commerce, entertainment, and writing, among others. Overall, these responses indicate that conversational AI agents are already widely used in numerous domains, suggesting a growing reliance on AI technology for various tasks.

Figure 1 (top) shows the percentage of participants who indicated that they would like to see some kind of affective empathy, which includes any of the three abilities (i.e., perceiving, responding to, or simulating emotions). As can be seen, there is large variability across applications with some consistency within categories. For instance, only around 50% of the participants expressed a preference for affective empathy when enhancing coding accuracy and speed (A6), but 84% favored its incorporation when producing personalized email messages and responses (A14). Interestingly, around 80% of the respondents expected to see some form of emotional skills in AI agents for a large number of applications (50%),

The most preferred categories for affective empathy were found to be language and communication, emotional support and mental health (84.5% $\pm$ 1.5%), and creativity and content generation (80.8% $\pm$ 3.2%). These categories align with the inherently human aspects of interaction, empathy, and creativity, where the presence of affect in AI agents could be perceived as more beneficial. Conversely, the least preferred categories for affective empathy were data analysis and research (59.5% $\pm$ 1.5%), as well as productivity and time management (63% $\pm$ 8.6%). In these categories, participants may prioritize the efficiency and accuracy of AI agents over agents’ ability to demonstrate affective skills, viewing affective empathy as less essential to the tasks at hand.

As part of the survey, we incorporated open-ended questions to explore potential areas where affective AI agents might be advantageous or disadvantageous. The thematic analysis of beneficial scenarios revealed preferences for mental health and emotional support (27.7%), healthcare and wellness (21.9%), customer service and experience (20.7%), coaching and parenting (20.1%), content creation and language education/translation (10.7%), personal assistants and productivity (9.3%), entertainment and gaming (3.9%), and personalized recommendations (3.8%). For the most prominent theme on mental well-being and support for individuals, respondents emphasized applications that assist individuals in various social contexts, such as therapy and counseling, as well as applications that offer companionship for those experiencing loneliness, isolation, or challenging life situations.

The thematic analysis of non-beneficial cases uncovered a mix of less-than-ideal scenarios and general concerns related to the concept of affective empathy. The distribution of themes included scenarios that require a high level of objectivity (32.5%), those necessitating emotional support and healthcare in general (18.6%), situations demanding emotional authenticity and human interaction (17.2%), general privacy and security concerns associated with emotional data (7.5%), marketing and customer service scenarios due to potential manipulation (5.8%), and general opposition against emotional capabilities (5.5%). For the most prominent theme of objectivity, respondents provided examples that were primarily technical, data-driven, or objective, such as coding, data analysis, and research.

Figure 1 (bottom) shows the percentage of users who preferred to interact with AI agents having the ability to perceive (black), respond to (grey), and simulate (white) emotions across all the applications. The graph reveals that people generally expect AI agents to possess the ability to perceive emotions (66.33% $\pm$ 12.07% on average), followed closely by the ability to respond to emotions (61.87% $\pm$ 12.86%). Approximately half of the respondents expect AI agents to be capable of simulating emotions (35.47% $\pm$ 12.45%), although there is noticeable variance across settings. For certain applications, such as producing personalized email messages and responses (A14), providing effective coaching and mentorship (A18), providing critical crisis intervention and support (A29), and offering effective companionship and emotional support (A30), the scores for perceiving and responding to emotions were nearly identical.

Participants expressed a preference for AI agents that perceive and respond to emotions in applications involving human interaction or emotional support. Conversely, they expressed a preference for agents that can simulate emotions in creative or content generation tasks and emotional support. A lower preference for affective empathy, particularly for simulating emotions, was observed in data-focused or analytical tasks. Overall, the desired level of affect in AI agents appears to be highly dependent on the context and nature of the application. This underscores the importance of tailoring AI agent capabilities to meet the specific requirements and expectations of users across various domains.

When examining the open-ended questions, respondents shared diverse perspectives on the expected capabilities of affective AI agents. They acknowledged the potential benefits of AI agents that can accurately perceive and interpret users’ emotional states, but also expressed concerns about privacy, data security, and the possibility of misinterpretation or emotional biases. Respondents also highlighted the advantages of AI agents capable of simulating emotional expressions to create more engaging, human-like, and relatable interactions. However, they raised concerns about authenticity and sincerity, as well as the potential for simulated empathy to be perceived as manipulative, disingenuous, or even harmful. They worried that AI agents could exploit users’ emotional vulnerabilities. Interestingly, the contrasting views of benefits and concerns in areas such as mental health, emotional support, and customer service demonstrate the intricate and nuanced nature of public sentiment toward affective AI agents.

When evaluating preferences, respondents were asked to consider the application in which affective empathy made the most sense to them. Considering this application, Figure 2 (left) shows the preferred platform for interacting with AI agents. In this case, participants showed a significantly higher preference for desktop platforms and smartphones than other devices such as smartwatches and earphones ($\chi^{2}(1)~{}=~{}938.57$, $p<0.001$). Similarly, Figure 2 (center) shows the signals that participants would be happy to share to enable affective empathy as well as the percentage of participants who selected them. As can be seen, the results indicate a significant preference for sharing text and voice data over other modalities such, as facial expressions, body language, and physiological signals ($\chi^{2}(1)~{}=~{}655.14$, $p<0.001$). Finally, Figure 2 (right) shows the preferred communication channels. As can be seen, people preferred significantly more text-based interactions (messages) and audio (speaking) interactions than face-to-face interactions ($\chi^{2}(1)~{}=~{}465.36$, $p<0.001$).

These findings show some important preferences in terms of platforms, user signals, and communication channels when designing affective AI agents. Our survey did not allow us to delve much deeper into why these preferences emerged, as we were unaware of them before this survey. We intend to follow up with future research on device/signal preferences. Still, by being aware of and potentially accommodating these preferences, developers can create more preferred, engaging, and affective agents in the future.

Figure 3 shows the distribution of PAC scores across all participants, revealing large variance among individuals. For instance, around 30 participants did not select any affective skill across all applications, whereas around 220 people selected at least one empathy preference for all the applications.

To understand whether personal factors could explain PAC scores, we performed a linear mixed effects models in which we used the PAC scores as the dependent variable, and both personality and emotional experience information as the independent variables. All dependent variables were z-scored to ensure that a consistent scale was met across them. Table II shows the coefficients and statistical values, showing that multiple variables significantly contributed to the preference score. In particular, individuals with higher cognitive reappraisal skills contributed more significantly to a higher PAC score, suggesting that people who are able to reappraise their feelings effectively are more likely to prefer affective empathy. Similarly, higher ratings in difficulty identifying feelings decreased PAC scores, although less significantly. In terms of personality, participants ranking high in extroversion and conscientiousness and, in a lesser amount, agreeableness, contributed positively to the score. This finding indicates that people who score high on these factors tend to prefer affective empathy on AI agents.

Finally, we examined the correlation between PAC scores and the likelihood of expressing concerns on the open-ended questions. As expected, we found that the opposition to the agent having affective capabilities was significantly and negatively correlated with users’ preferences for affective capabilities overall ($r$=-0.36), similarly with perceiving ($r$=-0.19), responding to ($r$=-0.28), and simulating ($r$=-0.24) capabilities. Similarly, concerns about emotional authenticity and human interaction were significantly and negatively correlated with the preferences for simulating capabilities ($r$=-0.19).