Generate, Discriminate and Contrast:
A Semi-Supervised Sentence Representation Learning Framework

The training process of the generator/discriminator model is illustrated in Figure 1.(a). The model is trained to perform two tasks, i.e., generation and discrimination, simultaneously with NLI data. For the generation task, we first obtain two training instances for each NLI triplet $\{x_{ori},x_{entail},x_{contra}\}$. In particular, as for entailment, we place $x_{ori}$ into the pre-defined entailment input templates $T_{e}$ to obtain model input $T_{e}(x_{ori})$, and set the output label as the corresponding entailment hypothesis $x_{entail}$. Similarly, we use $T_{c}$ in the contradiction generation to obtain the training instance: $\{T_{c}(x_{ori}),x_{contra}\}$. The generator is trained to predict the output labels given the input prompts and sentences, as shown in the top of Figure 1.(a).

For the discrimination task, we apply the prompt template $T_{d}$ on the concatenated sentence pairs to obtain model inputs, and the output is either true or false, leading to two training instances: $\{T_{d}(x_{ori},x_{entail}),\text{true}\}$ and $\{T_{d}(x_{ori},x_{contra}),\text{false}\}$. The output is mapped from the annotated labels, i.e., entailment $\rightarrow$ true, and contradiction $\rightarrow$ false.

By adding the prompts, both the generation and discrimination tasks can be transformed into conditional generation tasks, which largely reduce the model complexity. The model can be trained with a standard conditional generation loss to maximize the probability of the output sequence $y_{1,...,M}$ given the input sequence $X$:

$$\begin{split}P(y_{1},&y_{2},...,y_{M}|X)=\\ &\prod_{m=1}^{M}P(y_{m}|y_{0},...,y_{m-1},X),\end{split}$$

(1)

where $y_{0}$ is the decoder start token.

To better balance the performance between the generation and discrimination tasks, we equally mix the generation and discrimination training instances, and employ a weighted sum of the generation perplexity and discrimination accuracy on the development set for model selection.

The data synthesis and quality check process is shown in Figure 1.(b). Given an unlabelled sentence $u$, we first augment it with the generation prompts to obtain model inputs $T_{e}(u)$ and $T_{c}(u)$, which are fed into the trained generator to get entailment/contradiction prediction $u_{entail}$ and $u_{contra}$. The generated triplets $\{T_{d}(u_{ori},u_{entail}),\text{true}\}$ and $\{T_{d}(u_{ori},u_{contra}),\text{false}\}$ are evaluated by the discriminator for quality check, where the output probabilities of ’true’ and ’false’ are compared with a predefined threshold $\alpha$ to filter out the low-quality triplets. Finally, we can get a clean synthetic corpus with both positive and negative pairs for sentence representation learning.

Contrastive learning has been widely used in sentence representation learning, achieving state-of-the-art performance on the STS benchmark (Gao et al., 2021; Ni et al., 2021). Recently, PromptBERT (Jiang et al., 2022) utilizes prompts to obtain sentence embeddings of much higher quality from encoder-based models. Inspired by it, we propose an improved prompt-based contrastive sentence representation model for text-to-text models.

Our prompt-based contrastive learning is shown in Figure 1.(c). we first augment the input sentence [X] with prompt $T_{s}$ shown in Table 1, and feed the "[PAD]" token into the decoder. We take the first decoder output as the sentence embedding as in (Ni et al., 2021). During training, we regard the entailment pairs as positive pairs, and the contradictory sentences as hard negative pairs. We also use in-batch negative sampling to include more negative samples during training. For a batch with $N$ triplets, the prompt-based contrastive learning loss function is defined as:

$$\mathcal{L}=\frac{e^{{\rm sim}(h_{i},h_{i}^{+})/\tau}}{\sum_{j=1}^{N}(e^{{\rm sim}(h_{i},h_{i}^{+})/\tau}+e^{{\rm sim}(h_{i},h_{i}^{-})/\tau})},$$

(2)

where $i,j$ is the sentence index, $\tau$ is the temperature hyper-parameter, ${\rm sim}(\cdot)$ is the cosine similarity function, and $\{h,h^{+},h^{-}\}$ are representations of the premise, and corresponding entailment and contradiction hypotheses.

We consider three different settings, including universal sentence embedding, domain adaptation and QA training.

Universal sentence embedding: We consider the general case where we have large-scale open domain unlabeled sentences and restricted amount of human-labeled NLI data. In this case, the GenSE is trained with a two-stage schema. In particular, we first train the sentence embedding model on the synthetic open-domain triplets from the generator/discriminator model, which is subsequently finetuned on the labeled NLI datasets.

Domain adaptation: The GenSE can also be used for domain adaptation. In this case, we assume the GenSE would have access to extra in-domain unlabeled data from the downstream tasks, which is regarded as a standard setting in unsupervised sentence embedding (Wang et al., 2021). The open-domain corpus are available as well. We follow the same procedure in Section 2.1 to obtain in-domain sentence triplets by taking the in-domain sentences as the input to the trained generator/discriminator model. Afterwards, the prompt-based sentence embedding model is trained in three stages: 1) Pretraining on open-domain synthetic pairs; 2) Adapting the sentence embedding to a specific domain using in-domain synthetic pairs; 3) Finetuning on labeled NLI data.

QA training: It is also not uncommon to use additional QA data to improve sentence representation learning. Here, we also develop a setting to demonstrate whether GenSE can further boost the performance along with labeled QA data. Since hard negatives are not available in QA data, we remove them from the loss function:

$$\mathcal{L}^{\prime}=\frac{e^{{\rm sim}(h_{i},h_{i}^{+})/\tau}}{\sum_{j=1}^{N}e^{{\rm sim}(h_{i},h_{i}^{+})/\tau}}.$$

(3)

We consider two ways to train the prompt-based contrastive learning model to demonstrate the effectiveness of GenSE: 1) GenSE-QA: the model is first trained on QA data and then finetuned on NLI data. 2) GenSE+: the model is first pretrained on the open-domain synthetic data generated by the generator/discriminator model, and then trained on the QA data, and finally finetuned on the NLI data.

We evaluate GenSE on seven popular STS tasks for universal sentence embedding, and on four datasets from various domains, including AskUbuntu (Lei et al., 2016), CQADupStack (Hoogeveen et al., 2015), Twitter (Xu et al., 2015; Lan et al., 2017), and BIOSSES (Soğancıoğlu et al., 2017). For all the experiments, we assume the only available labeled sentence pairs are NLI (MNLI+SNLI) (Bowman et al., 2015; Williams et al., 2018). For open-domain data synthesis, we sample sentences from C4 news-like and English partitions (Raffel et al., 2020) ²²2huggingface.co/datasets/c4, and obtain around 61M synthetic triplets. In the domain adaptation setting, we follow TSDAE (Wang et al., 2021) to use unlabelled training set as in-domain sentences for AskUbuntu, CQADupStack, and Twitter. Since no training set is available for BIOSSES, we use PubMed subset in the Pile (Gao et al., 2020) as in-domain sentences, and remove the sentences existing in the test set.

As for the QA training, we utilize public available QA data for training, since CommQA used in (Ni et al., 2021) is not released. We choose datasets that are sampled from web sources, and have a sentence as both input and output. We also remove datasets that are closely related to downstream tasks, e.g., Stack Exchange, as well as the ones that are manually annotated, e.g., MS MARCO (Bajaj et al., 2016), for fair comparison. Finally, we obtain 4M QA pairs, including ELI5 (Fan et al., 2019), GOOAQ (Khashabi et al., 2021), and Yahoo (Zhang et al., 2015).

We build GenSE upon the widely-used T5 Encoder-Decoder models (Raffel et al., 2020). For generator/discriminator training, we set the learning rate to 5e-5 and batch size to 256. The generator/discriminator is trained for 10 epochs with evaluation step set as 500, and $(accuracy-10\times ppl)$ as validation metric for model selection. For data synthesis, we use nucleus sampling (Holtzman et al., 2019) with $p=0.9$, and set the confidence threshold $\alpha=0.9$. For contrastive sentence representation learning, we set the learning rate to 5e-5 and batch size to 512 on NLI, and learning rate of 5e-5 with batch size of 1024 on the synthetic data. The temperature $\tau$ is set to 0.01 for QA training, and 0.05 for others.

We compare GenSE with T5-Base to previous state-of-the-art sentence embedding methods, including encoder-based and encoder-decoder based models. All baselines are trained on the labeled NLI corpus. We also include large-scale models and models trained with additional large-scale semi-structured CommQA data for comparison. The results are reported in Table 2.

Overall, GenSE can greatly outperform previous state-of-the-art models. Specifically, GenSE achieves an average Spearman’s correlation of 84.78, significantly outperforming Base-size sentence embedding models, and even surpassing methods with much larger model sizes, e.g., ST5-Enc-3B and ST5-Enc-3B-CommQA. GenSE attains new state-of-the-art base-model results on 6 out of 7 STS datasets, i.e., except the SICK-R tasks, demonstrating that our synthetic data can greatly improve sentence embedding quality with GenSE.

We also report the performance of GenSE+, which achieves even higher performance with an average Spearman’s correlation of 85.19 by integrating additional QA data. The results suggest that our synthetic data is complementary to semi-structured data like question-answer pairs. But compared to semi-structured data which requires lots of efforts for data mining and cleaning, our GenSE only need unlabelled sentences that are much easier to collect for various domains, exhibiting better practical applicability.

Direct transfer: We first consider the direct transfer scenario where unlabeled in-domain data are not used for sentence embedding training, with results in the top of Table 4. We can see that GenSE shows 1.9 % improvements over strong ST5 baselines, and GenSE-QA works even slightly better. Combining QA and synthetic data, i.e., GenSE+, leads to a substantial improvement of 3.2%, demonstrating the effectiveness of synthetic data and QA pairs.

Domain adaptation: We also evaluate the model for domain adaptation, where in-domain synthetic data is also included in contrastive learning. We can observe that the average performance is improve by 1.7% over the best TSDAE-BERT-Base model through using in-domain data. The consistent and significant improvements over the four tasks also demonstrate the great generalization ability across various domains of GenSE. Furthermore, compared to direct transfer, we can also achieve an improvement of 0.4% even without labeled QA data, which convincingly demonstrates that GenSE can make full use of unlabeled sentences for better sentence embedding.

In this section, we present ablation studies on each component in GenSE, including the prompt learning, model scale, amount of labeled or synthesized data, and synthesis strategy.

Prompt learning: It has been shown that adding prompt can improve the encoder-based sentence embedding (Jiang et al., 2022), but the impact of prompt in text-to-text sentence embedding remains unknown. We compare the performance of models trained on NLI with or without prompt. As shown in Table 4, adding prompt consistently improve the performance for both STS-B and transfer tasks across different model sizes. Ablation experiments on T5-Small show that stacking a single decoder cannot further improve the performance. Yet, adding prompt can further boost the performance. We hypothesize that prompt helps to elicit knowledge contained in the decoder, leading to performance improvement.

Model scale: We investigate whether GenSE works across different model scales. Due to resource limitation, we conduct experiments on T5-Small. Specifically, we first train a unified data generator from T5-Small. Then we use the model for data synthesis, which results in 34M training pairs. Finally, we fine-tune a T5-Small sentence embedding model with synthetic and NLI data. Table 5 shows the result. GenSE outperforms ST5-Small by 1.6% on average for STS tasks, and 1.1% for transfer tasks. Yet, compared to T5-Base, the performance gain for T5-Small is less significant. It’s reasonable since the small data augmenter is less expressive than base one, which produces less synthetic data with lower quality from same amount of unlabelled sentences. We also find that although adding decoder improves the sentence embedding for base and large size models, it brings no benefit for T5-small model. One possibility is that the T5-small decoder architecture is too shallow, which acts like a mean pooling in encoder-only model.

Synthetic data amount: We also study how the amount of synthetic data influence the final performance. As shown in Table 6, by adding 20% data, GenSE already achieves substantial improvement. GenSE achieves the best performance on STS-B dev set (88.9) with more data, which even outperforms fine-tuned cross-attention T5-Base (88.02). For the transfer tasks, the average performance continues to improve as more data are used. The experiments validate the idea of using open-domain sentences to improve model generalization ability.

Supervised data amount: We investigate whether synthetic data can help reduce the annotation burden. We train another GenSE model with 50% randomly-sampled NLI data, with results in Table 7. Together with Table 4 we can find that the performance of baselines degrades significantly, i.e., 0.6% on STS-B and 1.4% on transfer tasks. However, using synthetic data can greatly alleviate the performance drop, and even outperforms baseline models trained on full NLI data.

Data synthesis strategy: We demonstrate the superiority of our data synthesis strategy by comparing to previous methods that utilize synthetic data for sentence embedding under the same semi-supervised setting, i.e.,back-translation (Wieting and Gimpel, 2018; Zhang et al., 2021b) and retrieval (Thakur et al., 2021a). The results are in Table 8. We can see that both retrieval-based method and back-translation can improve the performance, while our generator/discriminator based method can further significantly boost the performance on both tasks. Further experiments for GenSE without hard negatives demonstrates that synthesizing negative pairs plays an important role for the performance improvement, as it provides more information for contrastive learning.

Uniformity and alignment: We investigate the uniformity and alignment of GenSE, which capture the quality of produced sentence embedding:

$$\mathcal{L}_{align}=-\mathop{\mathbb{E}}_{s,s^{+}\sim p_{pos}}\Arrowvert f(s)-f(s^{+})\Arrowvert,$$

(4)

$$\mathcal{L}_{uniform}=\log\mathop{\mathbb{E}}_{s,u\overset{\mathrm{iid}}{\sim}p_{data}}e^{-2\Arrowvert f(s)-f(u)\Arrowvert},$$

(5)

where $p_{data}$ is the distribution of positive pairs, and $p_{pos}$ refers to all entailment pairs. Smaller $\mathcal{L}_{align}$ indicates shorter distance between sentence embeddings of positive pairs, and smaller $\mathcal{L}_{uniform}$ means that the embedding space is more uniform.

We follow the setting in  (Ni et al., 2021) to measure $\mathcal{L}_{uniform}$ on the whole STS-B test set, and $\mathcal{L}_{align}$ on sentence pairs in STS-B test set with correlation scores higher than 4. As shown in Figure 2, GenSE shows better alignment and uniformity than the SimCSE model based on RoBERTa-Base. Compared to the supervised ST5 baseline models, GenSE achieves much lower uniformity loss but larger alignment loss. We conjecture that GenSE obtains lower uniformity by learning from more synthesized unlabeled data, which might bring some noisy pairs and result in large alignment loss.

Quality of synthetic data: Several methods have been implemented to synthesize the sentence pairs, including generation-based DINO (Schick and Schütze, 2021), retrieval-based approach (Thakur et al., 2021a), and back-translation, with samples shown in Table 9. For each input sentences, we give two samples using over-generation. We run the experiment on the image caption data CC12M (Changpinyo et al., 2021). For DINO, we use the official repo ⁵⁵5https://github.com/timoschick/dino. For retrieval, we follow (Thakur et al., 2021a) to use BM25 (Amati, 2009) to produce possible pairs, and use a cross-encoder trained on NLI to further label the pair. For back-translation, we use google translator to produce English-French and English-German pairs.

For entailment pair generation, unsupervised DINO (Schick and Schütze, 2021) can generate some meaningful pairs with a large-scale generative model. It also produces many incorrect pairs, and cannot be filtered out since no supervision signal is available. BackTrans, Retrieval and GenSE can all produce entailments efficiently. However, BackTrans usually produces entailment pairs with very high lexical overlap, which fail to give high-quality supervision signals for contrastive representation learning. Retrieval can produce noisy pairs even after filtering due to the limited size of the banking corpus.

For contradiction pair generation, DINO hardly generate correct or related sentences, which cannot serve as hard negatives in sentence representation learning. Although retrieval and back-translation can produce entailment pairs, they cannot generate contradiction pairs. In contrast, GenSE can produce both entailment and contradiction pairs, which are important for contrastive sentence representation learning as demonstrated in Table 8.