Energy-based Unknown Intent Detection with Data Manipulation

Given a training dataset $\mathcal{D}^{\rm{train}}_{\rm{in}}=\{(\mathbf{u}^{(i)},y^{(i)})\}^{N}_{i=1}$ where $\mathbf{u}^{(i)}$ is an utterance and $y^{(i)}\in Y_{\rm{in}}=\{y_{1},y_{2},...,y_{K}\}$ is its intent label. In testing, given an utterance, unknown intent detection aims to detect whether its intent belongs to existing intents $Y_{\rm{in}}$. In general, unknown intent detection is an OOD detection task. The essence of all methods is to learn a score function that maps each utterance $\mathbf{u}$ to a single scalar that is distinguishable between IND and OOD utterances.

An energy-based model  LeCun et al. (2006) builds an energy function $E(\mathbf{u})$ that maps an input $\mathbf{u}$ to a scalar called energy score (i.e., $E:\mathbb{R}^{D}\rightarrow\mathbb{R}$). Using the energy function, probability density $p(\mathbf{u})$ can be expressed as:

where $Z=\int_{\mathbf{u}}\exp(-E(\mathbf{u})/T)$ is the normalizing constant also known as the partition function and $T$ is the temperature parameter. Take the logarithm of both side of (1), we can get the equation:

Since $Z$ is constant for all input $\mathbf{u}$, we can ignore the last term $\log Z$ and find that the negative energy function $-E(\mathbf{u})$ is in fact linearly aligned with the log likelihood function, which is desirable for OOD detection  Liu et al. (2020).

The energy-based model has a connection with a softmax-based classifier. For a classification problem with $K$ classes, a parametric function $f$ maps each input $\mathbf{u}$ to $K$ real-valued numbers (i.e., $f:\mathbb{R}^{D}\rightarrow\mathbb{R}^{K}$), known as logits. Logits are used to parameterize a categorical distribution using a softmax function:

where $f_{y}(\mathbf{u})$ indicates the $y^{\mathrm{th}}$ index of $f(\mathbf{u})$, i.e., the logit corresponding the $y^{\mathrm{th}}$ class label. And these logits can be reused to define an energy function without changing function $f$  Liu et al. (2020); Grathwohl et al. (2020):

According to the above, a classifier can be reinterpreted as an energy-based model. It also means the energy score can be derived from any classifier.

Due to its consistency with density and accessibility, we introduce the energy score for unknown intent detection, and utterances with higher energy scores can be viewed as OOD. Mathematically, the energy-based detector $G$ can be described as:

where $\delta$ is the threshold.

Although the energy score can be easily computed from the classifier, the energy gap between IND and OOD samples might not always be optimal for differentiation. To solve this problem,  Liu et al. (2020) propose an energy-bounded learning objective to further widen the energy gap. Specifically, the training objective of the classifier combines the standard cross-entropy loss with a regularization loss:

where $F(\mathbf{u})$ is the softmax output, $\lambda$ is the auxiliary loss weight. The regularization loss is defined in terms of energy:

which utilizes both labeled IND data $\mathcal{D}^{\rm{train}}_{\rm{in}}$ and auxiliary unlabeled OOD data $\mathcal{D}^{\rm{train}}_{\rm{out}}$. This term differentiates the energy scores between IND and OOD samples by using two squared hinge loss with the margin hyper-parameters $m_{\rm{in}}$ and $m_{\rm{out}}$.

Ideally, one has to sample all types of OOD utterances to create the gap, which is impossible in practice.  Zheng et al. (2019) demonstrate that OOD utterances akin to IND utterances could be more effective, but more difficult to collect. To address this problem, we propose a data manipulation framework, which can generate these high-quality OOD utterances and assign each generated utterance an importance weight to reduce the impact of potential bad generation.

Since not all words in utterances are meaningful, such as stop words, when generating OOD utterances, replacing these words may not change the intent. It is more efficient and effective to replace those intent-related words. Hence, we design an intent-related score function $S$ to measure how a word $w$ related to an intent $y$:

where $\mathcal{D}^{\rm{train}}_{y}$ is the subset of $\mathcal{D}^{\rm{train}}_{\rm{in}}$, which contains utterances with intent $y$, $\mathbb{I}$ is the indicator function, $w_{j}$ is the $j^{\rm{th}}$ word in $\mathbf{u}$, and $\boldsymbol{w}_{<j}=w_{1},...,w_{j-1}$.

Given $w$ and $y$, the intent-related score function is the sum of the log-likelihood ratios for all $w$ in $\mathcal{D}^{\rm{train}}_{y}$. If $w$ is related to $y$, $w$ tends to occur more frequently in $\mathcal{D}^{\rm{train}}_{y}$ than other words. For each occurrence of $w$, i.e., $w_{j}$ equals $w$, $p(w|\boldsymbol{w}_{<j},y)$ should be higher than $p(w|\boldsymbol{w}_{<j})$ as the former is additionally conditioned on the related $y$, while the latter is not, hence resulting in a higher $S(w,y)$. In contrast, if $w$ is not related to $y$, $p(w|\boldsymbol{w}_{<j},y)$ is much less likely to be higher than $p(w|\boldsymbol{w}_{<j})$, or $w$ tends to have a lower frequency in $\mathcal{D}^{\rm{train}}_{y}$, hence $S(w,y)$ is likely to be small. Therefore, $S(w,y)$ can serve as a valid score function to measure how a word $w$ is related to an intent $y$.

With the help of $S$, given an utterance to be replaced and its intent label, the locating module calculates the intent-related score for each word in this utterance, and a word with a higher score (i.e., larger than a given threshold) can be viewed as an intent-related word.

After detecting intent-related words in the utterance $\mathbf{u}$, for each of the intent-related words $w_{t}$, the generating module aims to generate the replacement words from the vocabulary set to replace $w_{t}$ and obtain OOD utterances. We design a candidate score function $Q$ to measure the desirability of the candidate word $c$:

The first term of the right hand side is the log-likelihood of $c$ conditioned on the context of $w_{t}$; the higher it is, the more suitable $c$ is given the context. The second term of the right hand side is the negative log of the average likelihoods of $c$ conditioned on the IND label and previous context; the higher it is, the less relevant $c$ is to IND utterances. Therefore, if $c$ has a higher candidate score, that means it fits the context well and has a low density under the IND utterance distribution, thus can be selected as the replacement word to replace $w_{t}$. The resulting generated OOD utterance is:

Since we cannot ensure the generation process is perfect, given a generated OOD utterance set $\mathcal{D}^{\rm{gen}}_{\rm{out}}=\{\hat{\mathbf{u}}^{(i)}\}^{M}_{i=1}$, there might be some unfavorable utterances that are useless or even harmful for tuning the classifier. To fit these utterances, the generalization ability of the classifier will decrease. The weighting module aims to assign these utterances small weights.

We first use Equation  6 as the loss function to train a classifier by taking $\mathcal{D}^{\rm{gen}}_{\rm{out}}$ as $\mathcal{D}^{\rm{train}}_{\rm{out}}$. Then we calculate the influence value $\phi\in\mathbb{R}$  Wang et al. (2020) for each generated utterance $\hat{\mathbf{u}}$. The influence value approximates the influence of removing this utterance on the loss at validation samples. An utterance with positive $\phi$ implies that its removal will reduce the validation loss and strengthen the classifier’s generalization ability, thus we should assign it a small weight. ^***Details about how to calculate the influence can be found in  Koh and Liang (2017); Wang et al. (2020). In particular, given $\phi$, we calculate weight $\alpha$ as follows:

where $\gamma\in\mathbb{R}^{+}$ is used to make the weight distribution flat or steep, $\max_{\phi}$ and $\min_{\phi}$ are the maximum and minimum influence value of utterances in $\mathcal{D}^{\rm{gen}}_{\rm{out}}$.

We summarize the process of GOT  in Algorithm  1. Line 4 shows that $w_{j}$ can be viewed as an intent-related word for $y$ if $S(w_{j},y)$ is greater than the intent-related word threshold $\epsilon$. Line 5 shows that we generate $K$ replacement words with the top-$K$ $Q(c;\mathbf{u},w_{t})$.

After obtaining weighted OOD utterances set $\mathcal{D}^{\rm{gw}}_{\rm{out}}$, we can explicitly shape the energy gap with them, resulting in IND utterances with smaller energy scores and OOD utterances with higher energy scores. Specifically, we redefine the regularization loss in Equation  6 as follows and use it to re-train the classifier:

In the testing process, we can calculate the energy score for the utterance by Equation  4, and identify whether it is OOD by Equation  5.

To evaluate the effectiveness of the energy score and our proposed framework, we conducted experiments on two public datasets:

• •

  CLINC150^†††https://github.com/clinc/oos-eval (Larson et al., 2019): this dataset covers 150 intent classes over ten domains. It supports some OOD utterances that do not fall into any of the system’s supported intents to avoid splitting unknown intents manually.

• •

  SNIPS^‡‡‡https://github.com/snipsco/nlu-benchmark (Coucke et al., 2018): this dataset is a personal voice assistant dataset that contains seven intent classes. SNIPS does not explicitly include OOD utterances. We kept two classes SearchCreativeWork and SearchScreeningEvent as unknown intents.

Table  1 provides summary statistics about these two datasets. Note that the training set and validation set do not include OOD utterances.

We used four common metrics for OOD detection to measure the performance. AUROC (Davis and Goadrich, 2006), AUPR In and AUPR Out (Manning et al., 1999) are threshold-independent performance evaluations and higher values are better. FPR95 is the false positive rate (FPR) when the true positive rate (TPR) is 95%, and lower values are better.

Considering the smaller proportion of OOD utterances in the test set on two datasets, AUPR Out is more informative here.

We introduce the following classifier-based methods as baselines:

• •

  MSP (Hendrycks and Gimpel, 2017) trains a classifier with IND utterances and uses the softmax confidence score to detect OOD utterances.

• •

  DOC (Shu et al., 2017) trains a binary classifier for each IND intent and uses maximum binary classifier output to detect OOD utterances.

• •

  Mahalanobis (Lee et al., 2018) trains a classifier with softmax loss and uses Mahalanobis distance of the input to the nearest class-conditional Gaussian distribution to detect OOD utterances.

• •

  LMCL (Lin and Xu, 2019) uses LOF (Breunig et al., 2000) in the utterance representation learned by a classifier. In training, they replace the softmax loss with LMCL (Wang et al., 2018).

• •

  SEG (Yan et al., 2020) also uses LOF in the utterance representation. In training, they use semantic-enhanced large margin Gaussian mixture loss.

For a fair comparison, all classifiers used in the above methods and ours are pre-trained BERT (Devlin et al., 2018) with a multi-layer perceptron (MLP). We select parameter values based on validation accuracy. For energy score, we follow Liu et al. (2020) to set $T$ as 1, $\lambda$ as 0.1, $m_{\rm{in}}$ as -8 and $m_{\rm{out}}$ as -5. For influence value, we focus on changes on MLP parameters and use stochastic estimation Koh and Liang (2017) with the scaling term 1000 and the damping term 0.003. For LMCL implementation, we set nearest neighbor number as 20, scaling factor s as 30 and cosine margin m as 0.35, which is recommended by Lin and Xu (2019). For SEG, we follow Yan et al. (2020) to set margin as 1 and trade-off parameter as 0.5.

For our framework, we set candidate number $K$ as 2, weight term $\gamma$ as 20. In particular, for CLINC150, we set threshold $\epsilon$ as 150 and generate 100 weighted utterances for each intent. For SNIPS, we set threshold $\epsilon$ as 1500 and generate 1800 weighted utterances for each intent. The difference in settings between two datasets is due to the different sizes of per intent in the training set.

As shown in Table  2, we can observe that:

• •

  The energy score can achieve comparable results on two datasets. Note that on SNIPS dataset, the advantages of the energy score are not as obvious. The reason is that SNIPS dataset is not as challenging as CLINC150 dataset, most methods can achieve good results, such as AUPR out is greater than 0.9.

• •

  Energy + GOT  achieves better results on two datasets as compared to the raw energy score. It indicates that our generated weighted OOD utterances can effectively shape the energy gap, resulting in more distinguishable between IND and OOD utterances.

• •

  We also report ablation study results. “w/o weighting” is the energy score tuned by OOD utterances without reweighting. We can see that there is a decrease in performance on both datasets, which shows the advantage of the weighting module (p-value < 0.005).

During the training process, we find that the method performance is sensitive to two hyper-parameters: auxiliary loss weight $\lambda$ and the number of generated weighted OOD utterances per intent. We conduct two experiments to demonstrate their effects separately. We choose CLINC150 dataset as it is more challenging as mentioned before.

An easy way to obtain OOD utterances is from the Wikipedia corpus. We investigate the effect of regarding Wikipedia sentences as OOD utterances to shape the energy gap on CLINC150 dataset. The Wikipedia sentences are from  Larson et al. (2019) and the number is 14750.

As shown in Table  3, we can observe that these sentences cannot improve the performance and even have a negative effect (We experimented with several hyper-parameters, this is the best result we could get). After observing these Wikipedia sentences, we find that they have little relevance to IND utterances. Therefore, simply using Wikipedia sentences is unrepresentative and ineffective for shaping the energy gap.

As mentioned in Section  1, when using the softmax-based detector, OOD inputs may also receive a high softmax confidence score. To tackle this problem,  Lee et al. (2017) replace the cross entropy loss with the confidence loss. The confidence loss adds the Kullback-Leibler loss (KL loss) on the original cross entropy loss, which forces OOD inputs less confident by making their predictive distribution to be closer to uniform.

To verify the generality of GOT, we directly use the generated weighted OOD utterances to fine-tune the softmax-based detector with the confidence loss. The results are shown in Table  4. Our MSP + GOT has a significant improvement and outperforms MSP by 8.9% (AUPR Out). Figure  4 provides an intuitive presentation. The softmax confidence scores of OOD from MSP form smooth distributions (see Figure  4 (left)). In contrast, the softmax confidence scores of OOD from MSP + GOT concentrate on small values (see Figure  4 (right)). Overall the softmax confidence score is more distinguishable between IND and OOD after tuning by GOT.

We sample some intents and showcase generated weighted OOD utterances in Table 5. We can observe that intent-related words that located by our locating module are diverse, containing not only words appeared in the intent label. The replacement word fits the context well, and the intent of the generated utterance is exactly changed in most conditions. Admittedly, GOT may have a bad generation, like replace “benefits” with “services” in the second utterance, which leads the generated utterance is still in-domain. Fortunately, the weighting module assigns these utterances a lower weight to reduce their potential harm.