MIME: MIMicking Emotions for Empathetic Response Generation

As Rashkin2018IKT and MoEL, to explicitly infuse emotion into the context representation $c$, we train a emotion classifier on $c$. We train emotion embeddings $E_{\mathcal{E}}\in\mathbb{R}^{n_{emo}\times D_{h}}$ ($n_{emo}=32$ is the number of emotion classes) to represent each emotion. We maximize the similarity between $c$ and user-emotion representation $E_{\mathcal{E}}(e)$, $e$ being the user-emotion label, using cross-entropy loss $L_{cls}$:

	$\displaystyle s$	$\displaystyle=E_{\mathcal{E}}W_{\mathcal{E}}c^{T},$		(4)
	$\displaystyle\mathcal{P}$	$\displaystyle=\operatorname{softmax}(s),$		(5)
	$\displaystyle L_{cls}$	$\displaystyle=-\log\mathcal{P}[e],$		(6)

where $W_{\mathcal{E}}\in\mathbb{R}^{D_{h}\times D_{h}}$ and $s,\mathcal{P}\in\mathbb{R}^{n_{emo}}$.

We split the 32 emotion types into two groups containing 13 positive and 19 negative emotions, as listed in Table 1. This split is guided by our intuition.

There are more than one correct way to respond empathetically. However, we observed that the SOTA model, MoEL, often resorts to generic and repetitive, although empathetic, responses. Therefore, inspired by vhred, we introduce stochasticity in the response-emotion determination that resulted in emotionally more varied responses in our experiments. To this end, we sample response-emotion distributions $d_{pos}$ and $d_{neg}$, from the context $C$ — specifically, $c$ in Eq. 3 —, for both positive and negative emotion groups, respectively. Hence, we sample an unnormalized distribution $z_{g}$ ($g\in\{pos,neg\}$) from distribution $P_{\theta}(z_{g}|C)$. This $z_{g}$ is passed to a fully-connected layer ($\operatorname{FC}_{d_{g}}$) with softmax activation to obtain the normalized distribution $d_{g}\in\mathbb{R}^{n_{g}}$ ($n_{pos}=13$ and $n_{neg}=19$):

	$\displaystyle P_{\theta}(z_{g}|C)$	$\displaystyle=\mathcal{N}(\mu_{g}^{\text{prior}}(C),\sigma_{g}^{\text{prior}}(C)),$		(7)
	$\displaystyle z_{g}$	$\displaystyle\sim P_{\theta}(z_{g}|C),$		(8)
	$\displaystyle d_{g}$	$\displaystyle=\operatorname{softmax}(\operatorname{FC}_{d_{g}}(z_{g})).$		(9)

The emotion representation for each emotion group, $e_{g}\in\mathbb{R}^{D_{h}}$, is obtained by pooling the corresponding emotion embeddings using the respective distribution $d_{g}$:

$\displaystyle e_{g}$

$\displaystyle=d_{g}E_{\mathcal{E}_{g}},$

(10)

where $E_{\mathcal{E}_{g}}\in\mathbb{R}^{n_{g}\times D_{h}}$ are emotion embeddings in the emotion group $g$ — as defined in Table 1.

Sampling from distribution $P_{\theta}(z_{g}|C)$ is reparameterized as follows:

	$\displaystyle c^{\prime}$	$\displaystyle=\operatorname{ReLU}(\operatorname{FC}_{sample}(c)),$		(11)
	$\displaystyle\mu_{g}^{\text{prior}}(C)$	$\displaystyle=\operatorname{FC}_{\mu_{g}}(c^{\prime}),$		(12)
	$\displaystyle\sigma_{g}^{\text{prior}}(C)$	$\displaystyle=\exp(0.5\operatorname{FC}_{\sigma_{g}}(c^{\prime})),$		(13)
	$\displaystyle r$	$\displaystyle\sim\mathcal{N}(0,I),$		(14)
	$\displaystyle z_{g}$	$\displaystyle=\mu_{g}^{\text{prior}}(C)+r\odot\sigma_{g}^{\text{prior}}(C),$		(15)

where $g\in\{pos,neg\}$, $\operatorname{FC}_{*}$ are fully-connected layers with output sizes $D_{h}$. Following vhred, $P_{\theta}(z_{g}|C)$ is obtained by maximizing evidence lower-bound ($-L_{g}^{ELBO}$):

$$L_{g}^{ELBO}=\operatorname{KL}[Q_{\psi}(z_{g}|e_{g},C)||P_{\theta}(z_{g}|C)]\\ -\mathbb{E}_{Q_{\psi}(z_{g}|e_{g},C)}[\log P_{\theta}(e_{g}|z_{g},C)],$$

(16)

where $Q_{\psi}(z_{g}|e_{g},C)$ is the approximate posterior distribution, defined as

$$Q_{\psi}(z_{g}|e_{g},C)\\ =\mathcal{N}(\mu_{g}^{\text{posterior}}(e_{g},C),\sigma_{g}^{\text{posterior}}(e_{g},C)),$$

(17)

which is similarly reparameterized, for sampling during the training only, as $P_{\theta}(z_{g}|C)$, except $e_{g}$ is concatenated to $c$.

Following Carr5497, it is reasonable to assume that the empathetic response to an emotional utterance would likely mimic the emotion of the user to some degree. Responding empathetically to positive utterances usually requires positivity — there can be ambivalence (Fig. 1). On the other hand, the responses to negative utterances should contain some empathetic negativity, but mixed with some positivity to soothe the user’s negativity. Thus, we generate two distinct response-emotion-refined context representations — mimicking and non-mimicking — that are appropriately merged to obtain response-decoder input.

Naturally, mimicking and non-mimicking emotion representations — $m$ and $\widetilde{m}$ — are defined as follows:

	$\displaystyle m$	$\displaystyle=e_{pos}\text{ if }e\text{ is positive, otherwise }e_{neg},$		(18)
	$\displaystyle\widetilde{m}$	$\displaystyle=e_{neg}\text{ if }e\text{ is positive, otherwise }e_{pos}.$		(19)

Firstly, response-emotion representations — $m$ and $\widetilde{m}$ — are concatenated to the context-enriched word representations in $H_{1:k+1}$ (Eq. 2) to provide the context ($C$) the cues on the response emotion:

	$\displaystyle H_{resp}$	$\displaystyle=[H_{i}\oplus m]^{k+1}_{i=1},$		(20)
	$\displaystyle\widetilde{H}_{resp}$	$\displaystyle=[H_{i}\oplus\widetilde{m}]^{k+1}_{i=1},$		(21)

where $H_{resp},\widetilde{H}_{resp}\in\mathbb{R}^{k\times 2D_{h}}$ are fed to a transformer encoder ($\text{TR}_{\text{Enc}}^{resp}$) to obtain mimicking and non-mimicking response-emotion-refined context representations $M$ and $\widetilde{M}$, respectively:

	$\displaystyle M$	$\displaystyle=\text{TR}_{\text{Enc}}^{resp}(H_{resp}),$		(22)
	$\displaystyle\widetilde{M}$	$\displaystyle=\text{TR}_{\text{Enc}}^{resp}(\widetilde{H}_{resp}),$		(23)

where $M,\widetilde{M}\in\mathbb{R}^{k\times D_{h}}$.

Enabling mixture of positive and negative emotions, inspired by lin-etal-2019-moel and the instance in Fig. 1, could lead to diverse response generation as compared to considering exclusively positive or negative emotion. To achieve this mixture, we concatenate $M$ and $\widetilde{M}$ at word level, as opposed to sequence level, to obtain $M^{\prime}\in\mathbb{R}^{k\times 2D_{h}}$. Then, $M^{\prime}$ is fed to a gate consisting of a fully-connected layer ($\operatorname{FC}_{contrib}$) with sigmoid activation, resulting $M_{contrib}$ that determines the contribution of postive and negative response-emotion-refined contexts to the response to be generated. Subsequently, $M^{\prime}$ is multiplied with the gate output, yielding the refined context $M_{adjust}$ that is fed to another fully-connected layer $\operatorname{FC}_{fused}$ to obtain the fused response-emotion-refined context $M_{fused}\in\mathbb{R}^{k\times D_{h}}$:

	$\displaystyle M^{\prime}$	$\displaystyle=[M_{i}\oplus\widetilde{M}_{i}]_{i=1}^{k},$		(24)
	$\displaystyle M_{contrib}$	$\displaystyle=\sigma(\operatorname{FC}_{contrib}(M^{\prime})),$		(25)
	$\displaystyle M_{adjust}$	$\displaystyle=M_{contrib}\odot M^{\prime},$		(26)
	$\displaystyle M_{fused}$	$\displaystyle=\operatorname{FC}_{fused}(M_{adjust}).$		(27)

For the final response generation from the response-emotion-refined context $M_{fused}$, following MoEL, a transformer decoder ($\text{TR}_{\text{Dec}}^{resp}$), with $H_{1:k+1}$ as key and $M_{fused}$ as value, is employed:

	$\displaystyle O=\text{TR}_{\text{Dec}}^{resp}(H_{1:k+1},M_{fused}),$		(28)
	$\displaystyle\mathcal{P}_{resp}=\operatorname{softmax}(\operatorname{FC}_{decode}(O)),$		(29)
	$\displaystyle p(R_{i}|C,R_{0:i-1})=\mathcal{P}_{resp}[i],$		(30)

where $O\in\mathbb{R}^{t\times D_{h}}$, $t$ is the number tokens in response $R$, $\operatorname{FC}_{decode}$ is a fully-connected layer of output size $|V|$ — also the vocabulary size —, $\mathcal{P}_{resp}\in\mathbb{R}^{t\times|V|}$, and $p(R_{i}|C,R_{0:i-1})$ is the probability distribution on each response token.

Finally, categorical cross-entropy quantifies the generation loss with respect to the gold response $R_{gold}$:

$\displaystyle L_{resp}=-\log p(R_{gold}|C).$

(31)

Following Rashkin2018IKT, a transformer encoder-decoder NIPS2017_7181 generates a response as the user emotion is classified from the encoder output — equivalent to $c$ in Eq. 3.

This state-of-the-art method lin-etal-2019-moel performs user-emotion classification as Multi-TR. However, in contrast to our method, it employs emotion-specific decoders whose outputs are aggregated and fed to a final decoder to generate the empathetic response.

Firstly, we randomly sample four instances of each of the 32 emotion labels from the test set, resulting in a total of 128 instances, compared to the 100 instances used for the evaluation of MoEL  lin-etal-2019-moel. Next, we ask three human annotators to rate each sub-sampled model response on a scale from 1 to 5 (best score) on three distinct attributes: empathy (How much emotional understanding does the response show?), relevance (How much topical relevance does the response have to the context?), and fluency (How much linguistic clarity does the response have?). Scores across 128 samples and three annotators are averaged to obtain the final rating.

Given two models A and B — in our case MoEL and MIME (our model), respectively — we ask three human annotators to pick the model with the best response for each of the 128 sub-sampled test instances. The annotators can select a Tie if the responses from both models are deemed equal. The final verdict on each instance is determined by majority voting. In case no two annotators agree on a selection – that is all three annotators reached three distinct conclusions: MoEL, MIME, and Tie – we bring in a fourth annotator. From this, we calculate the percentage of samples where A or B generates a better response and where A and B are equal.

Based on the results in Table 3, we note that the responses from MIME are more often preferable to humans than the responses from MoEL and Multi-TR. This correlates with the results in Table 2 that indicate better empathy and contextual relevance for MIME. The inter-annotator agreement was 0.33 of the kappa score (fair).

We observe (Table 4) that the responses generated by MIME for both positive and negative user emotions are generally better in terms of empathy and fluency. Interestingly, MoEL seems to perform better on responding to negative emotions than to positive emotions in terms of empathy and fluency. We posit this stems from the abundance of negative samples in the dataset as compared to positive samples — 13 positive and 19 negative emotions roughly uniformly distributed. This may suggest that MoEL is more sensitive to positive/negative context imbalance in the dataset than MIME and Multi-TR.

To assess the contribution of user-emotion mimicry, we disabled it by passing $e_{g}$ (Eq. 10) directly to Eqs. 20 and 21. This results in a substantial drop in empathy, by 0.2 as per Table 5. We delve deeper by plotting the emotion embeddings produced with and without emotion mimicry in Fig. 3(a) and Fig. 3(b), respectively. It is evident that the separation of positive and negative emotions clusters is much clearer with emotion mimicry than otherwise, suggesting better emotion understanding in the prior case through emotion disentanglement. On the other hand, we observe a slight increase of relevance, by 0.03. We surmise this is caused by the absence of the confounding effect of swapping the value of $m$ and $\widetilde{m}$, in Eqs. 18 and 19, depending on the user emotion type. This may coerce the same set of parameters to learn processing both positive and negative emotions.

Looking at Table 5, we observe a performance drop in both empathy and relevance, by 0.73 and 0.02, respectively, in the absence of emotion grouping. This indicates the importance of having positive and negative emotions treated separately, rather than huddling them into a single distribution. We posit that the latter case causes all the emotions to compete for the importance which may lead to emotion uniformity in some cases or one emotion-type overwhelming the other in other cases. This in turn may lead to emotionally mundane and generic responses.

Based on the comparative results for relevance shown in  (Table 2), MIME appears to generate responses that are a closer fit to the context than MoEL does. Fig. 4 shows a test instance where MIME pulls key information from the context — the word ‘interview’ — to generate an empathetic and relevant response. The response from MoEL is also empathetic, but somehow more generic. We surmise that this can be attributed to the two-way context flow through the emotion embedding sharing and encoder output, as discussed in Section 5.1.

Similarly, Fig. 5 shows a conversation with an apprehensive user who shares a frightening story with a positive outcome. Here, MoEL fixates on the initial negative emotion of the user and replies with an unwarranted negatively empathetic response. MIME, however, responds with appropriate positivity hinted at the last utterance. Moreover, it can correctly interpret the events described as a ‘beautiful memory’, which is truly empathetic and relevant. Again, a strong mixture of context and emotion, facilitated by the emotion embedding sharing, is likely to be responsible for this.

As evidenced by Table 2, MIME falters in fluency as compared to MoEL. Fig. 6 shows an instance where MoEL generates an empathetic, yet somewhat generic, and fluent response. In contrast, the first response utterance from MIME — “I would have been to the police” — does make contextual sense. However, the second utterance “I would be a little better” reads incoherent and semantically unclear. Perhaps the model meant something like ‘I would have felt a little safer’. We repeatedly observed such errors, leading to poor fluency. Given the empathy- and relevance-focused structure of our model, we think MIME focused on its learning on empathy and relevance, at the cost of fluency. We believe this issue could be mitigated with additional training samples.

In our experiments, we assumed the emotion surprised to be positive (Table 1), and thus MIME responds with positivity to most test instances incurring surprise as a user emotion. However, this is not accurate, as one can be both positive and negatively surprised — “I recently found out that a person I […] admired did not feel the same way. I was pretty surprised” vs “This mother’s day was amazing!”. We posit that re-annotating the instances with a negatively-surprised user with a new negative emotion, namely shocked, should help alleviate this issue significantly.

The {top-1, top-2, top-5} emotion-classification accuracies for MoEL are {38%, 63%, 74%}, as compared to {34%, 58%, 77%} for MIME. Since the emotion embeddings are shared between encoder and decoder in MIME, it supposedly also encodes some generation-specific information in addition to pure emotion as discussed in Section 5.1, thereby hindering the overall emotion-classification performance. Notably, MIME also performs well on top-5 classification. This is likely due to MIME’s ability to discern positive and negative emotion types — as indicated by Fig. 3(a) — that comes into prominence as you add more likely-labels into the consideration of top-$k$ classification by raising $k$.