TransESC: Smoothing Emotional Support Conversation via Turn-Level State Transition

Liu et al. (2021) propose the task of emotional support conversation and release the benchmark dataset ESConv. They append the support strategy as a special token into the beginning of each supportive response and the following generation process is conditioned on the predicted strategy token. Peng et al. (2022) propose a hierarchical graph network to utilize both the global emotion cause and the local user intention. Instead of using the single strategy to generate responses, Tu et al. (2022) incorporate commonsense knowledge and mixed response strategy into emotional support conversation. More recently, Cheng et al. (2022) propose look-ahead strategy planning to select strategies that can lead to the best long-term effects and Peng et al. (2023) attempt to select an appropriate strategy with the feedback of the seeker. However, all existing methods treat the dialogue history as a lengthy sequence and ignore the turn-level transition information that plays critical roles in driving the emotional support conversation in a more smooth and natural way.

Endowing emotion and empathy to the dialogue systems has gained more and more attentions recently. To achieve the former goal, both generation-based methods (Zhou et al., 2018; Zhou and Wang, 2018; Shen and Feng, 2020) and retrieval-based (Qiu et al., 2020; Lu et al., 2021) methods attempt to incorporate emotion into dialogue generation. However, it merely meets the basic quality of dialog systems. And to generate empathetic response, previous works incorporate affection (Alam et al., 2018; Rashkin et al., 2019; Lin et al., 2019; Majumder et al., 2020; Li et al., 2020, 2022), cognition (Sabour et al., 2022; Zhao et al., 2022) or persona Zhong et al. (2020) aspects of empathy. Intuitively, expressing empathy is only one of the necessary steps to achieve effective emotional support. By contrast, emotional support is a more high-level ability that dialogue systems are expected to have.

Our research is carried out on the Emotional Support Conversation dataset, ESConv (Liu et al., 2021). In each conversation, the seeker with a bad emotional state seeks help to go through the tough. And the supporter is supposed to identify the problem that the seeker is facing, console the seeker, and then provide some suggestions to help the seeker to overcome their problems. The support strategies adopted by the supporter are annotated in the dataset and there are eight types of strategies (e.g., question, reflection of feelings and providing suggestions). However, ESConv dataset does not contain keyword sets of each utterance and emotion labels ¹¹1We use 6 emotion categories: joy, anger, sadness, fear, disgust, and neutral. for the seeker’s turn, we leverage external tools to automatically annotate them. More details about annotation are provided in Appendix A.

Formally, let $D=[X_{1},X_{2},\cdots,X_{N}]$ denotes a dialogue history with $N$ utterances between the seeker and the supporter, where the $i$-th utterance $X_{i}=[w^{i}_{1},w^{i}_{2}\cdots,w^{i}_{m}]$ is a sequence of $m$ words. And each utterance is provided with the extracted set of top $k$ keywords $K_{i}=[k^{i}_{1},k^{i}_{2}\cdots,k^{i}_{k}]$. Besides, the adopted support strategy $S_{i}$ of the supporter and the emotional state label $E_{i}$ of the seeker are also available for the turn-level supervision. The goal is to generate the next utterance $Y$ from the stand of the supporter that is coherent to the dialogue history $D$ and supportive to reduce the seeker’s distress.

We adopt Transformer encoder (Vaswani et al., 2017) to obtain the contextual representations of the dialogue history. Following previous works (Tu et al., 2022), the dialogue is flattened into a word sequence. Then we append the special token $[\mathrm{CLS}]$ to the beginning of each utterance and another one for the upcoming response. And the context encoder produces the contextual embeddings $H^{c}\in\mathbb{R}^{N\times d_{h}}$.

In this section, we propose to grasp the turn-level transition information, including semantics transition, strategy transition and emotion transition, to explicitly smooth the emotional support and drive the conversation in a natural way. Specifically, we construct the state transition graph, with three types of state for each node and seven types of edges, to propagate and update the transition information. And all the three states are supervised at each dialogue turn to predict the keyword set of each utterance, the adopted strategy of the supporter and the emotional state of the seeker.

We utilize the turn-level annotation to supervise the transition information, driving the emotional support conversation in a smooth and natural way.

Finally, based on the vanilla Transformer decoder (Vaswani et al., 2017), we devise the transition aware decoder to inject the turn-level transition information into the process of response generation.

To make the generation process grounded on the selected strategy, we dynamically fuse the last strategy state $\hat{st}$ (the adopted strategy for the upcoming response) with the embeddings of the utterance sequence as the input of the decoder:

where $W^{1}\in\mathbb{R}^{2d_{h}\times d_{h}}$ and $b^{1}\in\mathbb{R}^{d_{h}}$ are trainable parameters and $E_{i}$ is the $i$-th embedding token of the response.

And for the emotion transition information, we dynamically combine it with the output of the context encoder $H^{c}$ to explicitly incorporate the emotional states of the seeker. Specifically, the emotion states $e_{i}$ of the seeker and commonsense knowledge $e^{oR}_{i}$ of the supporter, which is generated by the COMET model under the relation type oReact to imply what the emotional effect would exert on the seeker after the $i$-th utterance of the supporter, constitutes the emotional state sequence $H^{emo}$.

where $W^{2}\in\mathbb{R}^{2d_{h}\times d_{h}}$ and $b^{2}\in\mathbb{R}^{d_{h}}$ are trainable parameters.

Thus, for the target response $Y=[y_{1},y_{2},\cdots,y_{M}]$, to generate the $t$-th token $y_{t}$, the hidden representation of it from the decoder can be obtained:

In the end, we dynamically inject semantics transition information via the fusion of the last semantics difference representation $\Delta_{i}$ (latent semantic information for the upcoming utterance) and the hidden representation $h_{t}$ of the $t$-th token:

where $W^{3}\in\mathbb{R}^{2d_{h}\times d_{h}}$ and $b^{3}\in\mathbb{R}^{d_{h}}$ are trainable parameters.

The distribution over the vocabulary for the $t$-th token can be obtained by a softmax layer:

where $D$ is the input dialogue history.

We utilise the standard negative log-likelihood as the response generation loss function:

A multi-task learning framework is adopted to jointly minimize the response generation loss, the semantic keyword, strategy and emotion loss.

where $\gamma_{1}$, $\gamma_{2}$, $\gamma_{3}$ and $\gamma_{4}$ are hyper-parameters.

We compare our proposed TransESC with the following competitive baselines. They are four empathetic response generators: Transformer Vaswani et al. (2017), Multi-Task Transformer (Multi-TRS) Rashkin et al. (2019), MoEL Lin et al. (2019) and MIME Majumder et al. (2020); and two state-of-the-art models on ESC task: BlenderBot-Joint Liu et al. (2021), GLHG Peng et al. (2022) and MISC Tu et al. (2022). More details of them are described in Appendix C.

To be comparable with baselines, we implement our model based on BlenderBot-small (Roller et al., 2021) with the size of 90M parameters. The window size $w$ of turn-level transition is 2. The hidden dimension $d_{h}$ is set to 300 and the number of attention heads in relation enhanced multi-head attention and emotion aware attention graph are 16 and 4. Loss weights $\gamma_{1}$, $\gamma_{2}$, $\gamma_{3}$ and $\gamma_{4}$ are set to 1, 0.2, 1 and 1, respectively. AdamW Loshchilov and Hutter (2017) optimizer with $\beta_{1}=0.9$ and $\beta_{2}=0.999$ is used for training. We vary the learning rate during the training process with the initial learning rate of 2e-5 and use a linear warmup with 120 warmup steps. And the training process is performed on one single NVIDIA Tesla A100 GPU with a mini-batch size of 20. For inference, following Tu et al. (2022), we also adopt the decoding algorithms of Top-$p$ and Top-$k$ sampling with $p$=0.3, $k$=30, temperature $\tau$=0.7 and the repetition penalty 1.03.

To explore the impact of three types of transition information, we remove the corresponding state representation with edges in the transition graph, the turn-level label prediction and the injection into the decoder. Besides, to explore the effect of turn-level transition process, we also discard it by predicting three states with the whole dialogue history.

As shown in Table 3, the ablation of any types of transition information can lead to a drop in the automatic evaluation results, demonstrating the effectiveness of each one of them. To be more specific, the ablation of the strategy transition (w/o Stra.Trans) causes the most significant performance drop. The reason is that selecting the proper strategy to support the seeker plays the most pivotal role in ESC. And the impact of emotion transition (w/o Emo.Trans) is relatively small. It may be attributed to the noise of annotated emotion labels and the generated emotional knowledge.

Moreover, when we remove the whole process of turn-level state transition, the significant performance drop verifies our contribution that grasping the fine-grained transition information can drive the ESC in a more smooth and natural way.

In Table 4, we show a case with responses generated by TransESC and two baselines. With the emotion transition and strategy transition, after several turns of comforting, TransESC senses the emotion state joy of the seeker and it is

time to offer helpful suggestions with the correct predicted strategy. And through semantics transition, it grasp the determination of the seeker to suggest him to have a try and encourage him to face the failure. By contrast, MISC and BlenderBot-Joint drive the conversation improperly, leading to the ineffective responses.

We adjust different lengths of transition window for a deeper analysis of the impact of transition information modeling. Results are shown in Table 5. The model with the transition window length of 2 achieves the best performance. On the one hand, capturing the transition information in the shorter window could not sufficiently comprehend dependencies of utterance transition in the dialogue history. On the other hand, much more redundant transition information may be incorporated by the model with longer transition window, which would weaken the performance of our model.