Towards Multi-Turn Empathetic Dialogs with Positive Emotion Elicitation

The person who starts the dialog is regarded as user, and the other one is regarded as agent. The goal of our task is to conduct empathetic dialog generation with positive emotion elicitation. There are two main characteristics of this task. Firstly, from the perspective of dialog goals, the agent should successfully elicit positive emotions from users through multi-turn dialogs. If the emotion state of users at the first dialog turn is negative or neutral, the agent should lead the dialog to a positive emotion state. If the initial one is positive, the agent should keep the emotion state to be positive or neutral. Secondly, from the perspective of emotional changes, the dialogue should be conducted in a natural, empathetic and gradual way.

In this work, we collect the dataset from natural dialogs of real web users on public websites, instead of through data annotation by crowdsourcing. The reason is that the empathy expressing of real users are more natural, and their chatting topics are more close to everyday life scenarios. We first collect Chinese dialogs from public social media and implement similar data cleaning process as done in Bao et al. (2020), which yielding a dataset containing 1.2 billion two-people dialog sessions. Then we introduce an ERNIE (Sun et al., 2019) based TextCNN (Kim, 2014) model to recognize the emotion of each utterance in dialogs. The detailed filtering procedures on the raw dataset are shown as follows:

1) The first utterance and the last utterances are from the same speaker who plays the role of user.

2) The probability of any negative or neutral or positive emotion in the first utterance is greater than 0.5. It helps us to improve the quality of emotion polarity information that is identified on this dataset.

3) The probability of any positive emotion in the last utterance is greater than 0.9. It also helps us to improve the quality of emotion related automatically-annotated information.

4) Delete dialogs with non-emotion related topics, such as renting, job hunting, blind date, which are not related to emotion eliciting but generally end up with positive utterance like "thanks" or "good" etc. (via keywords detection).

5) Delete dialogs with specific persons, institutions, address (being recognized with the use of Name Entity Recognition tools (Lample et al., 2016)) for privacy consideration.

6) Delete dialogs with offensive language (Kim, 2014) to decrease the probability of generating offensive responses.

Finally, we collect 819,391 dialogs that start with any possible negative or neutral or positive emotion and end with a positive emotion, which we called PosEmoDial. Its statistics is provided in Table 2.

To learn how agent-side speakers conduct successful positive emotion elicitation, we delete the last utterance (from the user-side speaker) of each dialog, and require the model to predict agent-side response at each turn.

We denote the context as $\mathnormal{\{u_{1},...,u_{n}\}}$, the ground-truth response as ${r}$, the generated response as ${r^{\prime}}$. For the sake of practicality, we treat the probability of the ${u_{1}}$ being emotionally positive ${p(pos|u_{1})}$ or negative ${p(neg|u_{1})}$ as the initial emotion state of the user-side speaker. For model training, we concatenate ${p(pos|u_{1})}$ and ${p(neg|u_{1})}$ with context and ground-truth response as the input.

Previous works have shown the effectiveness of Valence-Arousal-Dominance (VAD) Lexicons for emotional dialog generation (Zhong et al., 2019, Colombo et al., 2019, Zhong et al., 2020a, Li et al., 2020a). We further validate the high accordance between VAD score and emotion polarity obtained by a well-trained ERNIE2-TextCNN emotion classifier (Sun et al., 2019, Kim, 2014). Therefore, for the sake of token-level generation control and model efficiency, the lexicon-based VAD vectors rather than neural network-based utterance representation is selected for emotion representation in our approach. We utilize the latest and largest VAD Lexicon, the NRC_VAD by Mohammad (2018), where Valence, Arousal, and Dominance are represented by continuous values in 0-1, indicating Negative to Positive, Claim to Excited, and Submissive to Dominant respectively. This lexicon includes 20,000 English vocabularies and their corresponding 13,870 distinct Chinese vocabularies. However, as there are 30k BPE tokens for the PLATO-2 lexicon. To fill this gap, we extends the NRC_VAD to cover all the PLATO-2 lexicon.

We define Emotional Distance (ED) as emotional changes across different utterances. Specifically, we employ the VAD lexicon to calculate the distance between the user initial emotion state and the generated response via a 2-Norm function, as shown in Eq.(1).

the first term calculates the expected VAD value of word in $u_{1}$, where $|u_{1}|$ denotes the length of the first utterance $u_{1}$, $u_{1,j}$ is the j-th word in $u_{1}$, $\mathbf{o_{u_{1,j}}}\in\mathbf{R^{3}}$ is a 3-dim vector representing emotion associated with the word $u_{1,j}$ in VAD space. The second term calculate the expected VAD value of the generated word at time step $t$, where $\mathbf{s_{t}}=\mathbf{softmax(h_{t})}$ ($\mathbf{s_{t}}\in\mathbf{R^{|V|}}$) is a confidence of the system of generating words $w_{1},...,w_{|V|}$ at time $t$. $\mathbf{o_{w_{i}}}\in\mathbf{R^{3}}$ is the VAD vector of i-th word in the vocabulary $[\mathbf{o_{w_{1}}};...;\mathbf{o_{w_{|V|}}}]$.

With the help of emotional distance, the new loss function is designed to ensure smooth emotional transitions along with the whole dialog flow as follows.

The basic idea of the PEG loss is using emotional distance to control the emotion of response. The design process of PEG loss is described as follows:

1) If the user’s starting emotion state is positive, the emotional state of the response is expected to align with the starting emotion state to keep the positive emotion of user in the whole dialogue. The PEG loss is designed as $\sum_{t=1}^{T}ED_{t}$ , which will control the emotion of the response to be close to the starting emotion state, where $ED_{t}$ is the measurement of emotional distance between the starting utterance and the generated response at time step t as illustrated in Eq.(1).

2) If the user’s starting emotion state is negative, the response is expected to express empathy at the dialogue’s initial stage, and progressively transit to positive emotional state to elicit the user’s positive emotion. Therefore, the emotional distance is required to be progressively increased throughout the whole dialog.

In order to progressively increase the emotional distance, we further improve the PEG loss by introducing a novel controlling function $f$, named as Dialog Progress Function. The $f(\cdot)$ is defined as:

where $max\_turn$ is the maximum number of turns in dialog contexts, and it is set to 7 in our experiments. $|C|$ denotes the number of turns in dialog context at current time step. The $f(\cdot)$ value will transit from positive value to negative value as contexts get longer as shown in the middle part of Figure 2.

With the dialogue progress function, the PEG loss is redesigned as $\sum_{t=1}^{T}[f(|C|)\cdot{ED_{t}}]$. Then the emotion of the response will be controlled as follows:

• •

  At the dialogue’s initial stage, the emotional distance will be encouraged to be small. In other words, the emotion of response is controlled to align with the user’s starting emotion to express empathy.

• •

  At the dialogue’s latter stage, the emotional distance will be encouraged to be big because of the negative value of function $f(|C|)$ results in negative loss. In other words, the emotion of response is controlled to be different from the starting emotion of user, which will be positive.

• •

  At the whole dialogue stage, the emotional distance will be progressively increased from a small value to a big value because of the progressive transition of function $f(|C|)$. In other words, the emotion of response is controlled to express empathy at the dialogue’s initial stage, and progressively transit to positive emotional state to elicit the user’s positive emotion.

Finally, we use the probability of positive emotion of $u_{1}$ to combine the two kinds of the PEG loss as:

if a dialog starts with a positive emotion, $p(pos|u_{1})$ will be close to 1, and the first term will play a leading role. If a dialog starts with a negative emotion, $p(pos|u_{1})$ will be close to 0, and the second term will play a leading role. Otherwise, both will work.

The potential drawback of the PEG loss is that the emotion of generated responses is required to align with $u_{1}$ at the initial stage. Therefore, the higher the probability of negative $u_{1}$ is, the more likely the PEG loss will encourage the generation of negative words at the initial dialog stage. Sometimes the responses containing these words can be injurious and offensive to users.

To address this issue, we add a NER loss to penalize the generation of too negative words with too small VAD values. The NER loss will be activated when $u_{1}$ is negative to balance the negative effect of the PEG loss. The NER loss is defined as:

where the notation is the same as described in the above PEG loss section.

The objective of the PEGE model is to minimize the following integrated Positive-emotion-guided Empathetic Loss (PEGE Loss) $L_{pege}$:

where $L_{NLL}^{Baseline}$ denotes the NLL loss:

where $T$ is the length of the target response $r$ and $r^{\prime}_{<t}$ denotes previously generated words before time $t$.

The hyper parameter $\alpha$ and $\beta$ in Eq.(5) denote the weights of PEG and NER loss respectively. We set $\alpha=5$ and $\beta=2$ for our final model based on grid search experiments.

Automatic evaluation metrics. Though BLEU and DISTINCT are two traditional metrics (Li et al., 2016, Lin et al., 2019), they have long been argued against its efficacy in open-domain dialogue generation (Liu et al., 2016), and either BLEU or DISTINCT is less relevant to our task. We keep them mostly as a reference.

To evaluate the efficacy of our model, we define three novel metrics that we describe next to account for the positive emotion guidance capability and emotion empathy capability.

PEG-Score: a new metric on a scale of [0,3] to measure the positive emotion guidance capability. It rewards the positive emotion the user obtained in the last half of utterances, i.e., $U_{user}^{last}={\{u_{-2},u_{-4},...,u_{-n/2}\}}$, and calculate the adjust averaged VAD values of each word in $U_{user}^{last}$. Sum up the averaged VAD values to obtain the PEG-Score:

E-Score: a new metric on a scale of [-3,0] to measure the emotion empathy capability. It penalizes the emotional distance between the agent responses and the user starting utterance ($u_{1}$) in the first half utterances, i.e., $U_{agent}^{first}={\{u_{2},u_{4},...,u_{n/2}\}}$, and calculates the averaged VAD values of each word in $U_{agent}^{first}$. We also calculate the averaged VAD for each word in $u_{1}$ as the starting emotion state. Then we subtract the two values and get their absolute VAD values. Sum up the absolute VAD values to obtain the E-Score:

PEGE-Score: to balance the evaluation of positive emotion guidance and empathy, we sum up PEG-Score and E-Score to obtain the PEGE-Score (on a scale of [-3,3]):

Human evaluation metrics. We run crowd-sourcing tasks at the level of both utterances and dialogs. Three crowd-sourcing workers are asked to score the response/dialog quality with a value of 0 or 1, and the final score is determined through the majority voting. These criterias are provided as follows:

Coherence: As an utterance level metric, it measures if the response is fluent, relevant and consistent with the context.

Informativeness: As an utterance level metric, it evaluates if the response is informative.

Positive emotion guidance: As a dialog level metric, it evaluates if the agent successfully guides the users from a non-positive emotion state to a positive emotion state, or keep their positive emotion state unchanged.

Empathy: As a dialog level metric, it is only measured when the positive emotion guidance score is 1 (else 0). It measures if the agent expresses empathy towards the user before positive emotion guidance, or keep the positive user not change as the criteria for positive emotion guidance.

We select MoEL (Lin et al., 2019) and MIME (Majumder et al., 2020), two state-of-the-art baselines which solely introduce emotion as auxiliary information like our model in empathetic dialog generation tasks. PLATO-2 (1.6B) (Bao et al., 2020) and PLATO-2-FT (fine-tuned version of PLATO-2 (1.6B) on PosEmoDial) which hold similar structure as our model are also selected.

However, since both MoEL and MIME are trained on the English dataset Empatheticdialog (Rashkin et al., 2019), we retrain them on PosEmoDial. For the sake of comparability, the semantic word embeddings of MoEL and MIME are initialized with the PLATO-2 embeddings (2048 dimensions).

In multi-turn dialogue tasks, self-chat is a commonly used method to simulate human-bot conversations (Li et al., 2019, Roller et al., 2021), where a model plays the role of both partners in the conversation. For the sake of our task-specificity, we employ the original PLATO-2 model to play the role of the user. Because we want to simulate actual application scenarios as much as possible, a general "user" instead of an emotionally trained one is more appropriate. Accordingly, the candidate models will play the role of agent respectively.

The way to start the interactive conversation needs special attention. As pointed out by Roller et al. (2021) , if starting with ’Hi!’, partners tend to greet with each other and only cover some shallow topics in the short conversation. Therefore, we construct 100 sentences as the starting utterance of different dialogues. Each sentence provides a specific context from the user’s perspective, 33 of them are negative, 34 of them are neutral, and 33 of them are positive. The agent and "user" are required to perform self-chats given the context. There are 10 turns (20 utterances) in each dialog, including the input start utterance. We carry out automatic evaluation on the 100 self-chat logs and randomly select 50 conversations from 100 self-chat logs for human evaluation.

Automatic evaluation. Table 3 provides the automatic evaluation results for all the models. First, in terms of positive emotion elicitation, it shows that our model performs the best. Our model and PLATO-2-FT, which are fine-tuned on our PosEmoDial dataset, gain substantial improvements compared to PLATO-2. It indicates the effectiveness of our dataset for improving positive emotion elicitation capability. Moreover, when comparing our model with PLATO-2-FT, it can also be noted that the PEGE loss can provide an additional improvement on positive emotion guidance capability. Therefore, we conclude that our dataset and PEGE loss can work jointly to improve positive emotion guidance capability efficiently. Second, in terms of dialog empathy, our model gains the best performance as well. Our model’s significant advantage over the second-best model PLATO-2-FT verifies the effectiveness of our loss design towards empathy capability. MoEL and MIME, which are not pre-trained on the large-scale corpus, are less capable of generating appropriate responses, hurting their empathetic dialog capability and resulting in a slightly worse E-Score than PLATO-2 and PLATO-2-FT. These results confirm the efficiency of our model in positive emotion elicitation while ensuring dialog empathy.

Human evaluation. Table 4 provides the human evaluation results for all the models. Our model has significantly better performance on two task-specific metrics (positive emotion guidance and empathy), considerably better performance on the coherence metric, and comparable performance on the informativeness metric. By comparing our model with PLATO-2-FT, our model obtains around 52% improvements on P.E.G. and 63% improvements on Emp. This remarkable result demonstrates the effectiveness of our PEGE loss on positive emotion guidance and empathy capability. Our dataset PosEmoDial also shows its effectiveness in training emotional dialog model as PLATO-2-FT fine-tuned on PosEmoDial outperforms PLATO-2 with 44% improvements on P.E.G. and 46% improvements on Emp. By applying PEGE loss and PosEmoDial simultaneously, our model gains 119% improvements on P.E.G. and 138% improvements on Emp. over PLATO-2, which further verifies the mutual benefits of our PEGE loss and PosEmoDial dataset.

Moreover, the models which get better performance on human evaluation metrics P.E.G. and Emp. also get higher scores on automatic evaluation metrics, PEG-Score, E-Score, and PEGE-Score. This result indicates the reliability of our proposed automatic metrics. We also observe that 81.37% of dialogues that successfully guide the user towards positive emotion express empathy before emotion elicitation. It verifies our proposed dialog task’s rationality, i.e., expressing empathy before transit to positive emotion elicitation is crucial for building a human-like dialog system with emotion perception and expression capability.

To evaluate the effect of the PEG loss and NER loss, we delete them respectively or simultaneously to get $L_{ner}$, $L_{peg}$ and $L_{NLL}^{Baseline}$. We also eliminate the impact of PoSEmoDial by fine-tuning PLATO-2 and our model on 1M randomly selected dataset, denote as $D_{plato}$ and $D_{pege}$. Note that when $L_{NLL}^{Baseline}$ is applied, the model is equivalent to the settings of PLATO-2-FT.

Table 5 illustrates the results of the ablation study. Our model with PEGE loss gets the best performance, and the model with $L_{ner}$ gets bad performance on all metrics. With only NER loss ($L_{ner}$) remains, the model is more inclined to generate positive responses directly instead of conditioned on the user emotion state transition, which may not necessarily lead to positive feedback from users. This result is consistent with our real-world observations that the response to a negative statement with positive emotion directly without any emotional transition sometimes is inappropriate and even offensive. As the PEG loss $L_{peg}$ is designed with both positive emotion elicitation capability and empathy capability, $L_{peg}$ performs much better. However, without NER loss, the model with $L_{peg}$ will endure the risk of generating excessively negative responses, which may sometimes be unacceptable to users as well, and therefore bring no gain with positive emotion elicitation. The results suggest that all components in PEGE loss $L_{pege}$ are valuable and indispensable.

The comparison between $D_{plato}$ and $D_{pege}$ illustrates that our model is not data-dependent and can be generalized in other datasets since considerable improvements can be obtained on all three metrics even PEGE model is trained on randomly selected data. Meanwhile, PosEmoDial can actually facilitate model performance for both PLATO-2 and PEGE, validating its effectiveness in our task.