Scalable Prompt Generation for Semi-supervised Learning with Language Models

Figure 1 shows the overall pipeline of our proposed methods. Unlike the original PET pipeline with manual prompts and verbalizers, we use a prompt generation function to generate multiple automatic prompts. Each PLM with automatic prompts serves as a labeler model. We train each of these prompts $+$ automatic verbalizer models with a labeled dataset $\mathcal{T}$ in few-shot settings. With an input sequence $x_{t}\in\mathcal{T}$ and the given label $y_{t}$, we first use the prompt function $P$ to transform $x_{t}$ into a sequence $P(x_{t})$ with a MASK token. The verbalizer then maps the predicted word probability at the masked position to the label probability. For each PLM $m$, the predicted probability $p_{m}(y_{t}|x_{t})$ is defined as

$$p_{m}(y_{t}|x_{t})=\frac{\exp m(y_{t}|x_{t})}{\sum_{y^{\prime}\in Y}\exp m(y^{\prime}|x_{t})}$$

(1)

where $m(y|x)$ is the raw score of PLM $m$ in the masked position. After obtaining the probability, we minimize the cross-entropy loss $\mathcal{L}_{c}$ between $p_{m}(y|x)$ and $y$.

We apply trained labeler models to each sentence $x_{d}\in\mathcal{D}$ in the unlabeled dataset $\mathcal{D}$ and get the probability $p_{m}(y_{d}|x_{d})$ for each trained model. We then take the average of these probabilities from each trained model $m$ as the ground-truth probability,

$$p_{t}(y_{d}|x_{d})=\frac{1}{Z}\sum_{m\in M}p_{m}(y_{d}|x_{d})$$

where $Z$ is the total number of trained PLMs with different automatic prompts. Eventually, we fine-tune a final pre-trained language model $\mathbf{F}$ with a standard sequence classification head. We use the Kullback-Leibler (KL) divergence as our loss function. Given $p_{t}(y_{d}|x_{d})$ and the predicted probability $\hat{p}(y_{d}|x_{d})$ of the final classifier $\mathbf{F}$, the divergence loss $\mathcal{L}_{div}$ for this input is:

$$\mathcal{L}_{div}(x_{d})=\sum_{y^{\prime}\in Y}p_{t}(y^{\prime}|x_{d})\log\left(\frac{p_{t}(y^{\prime}|x_{d})}{\hat{p}(y^{\prime}|x_{d})}\right)$$

(2)

The final classifier $\mathbf{F}$ is then applied to the test set to obtain the results.

Schick and Schütze (2021) introduce diversity in their SSL pipeline by training several models with different manual prompts and applying them to softly label a large number of unlabeled datasets. The diversity between manual prompts brings consistent improvements. We observe that diverse knowledge learned by the language model is mostly introduced by the prompts rather than manual verbalizers, since in most datasets, they prepare only one manual verbalizer but multiple prompts for experimentation. Thus, we propose replacing manual prompts with multiple automatic prompts and using the same automatic verbalizer for all labeler models.

Several researchers have proposed methods to automate the prompt design process Liu et al. (2021c); Li and Liang (2021); Lester et al. (2021). In most of these methods, they insert the continuous trainable prompt tokens into the input sentence and learn the token embeddings during the training process. However, existing continuous prompt-based learning methods do not consider their application in the PET pipeline, which requires training several labeler models Schick and Schütze (2021), in order to learn diverse knowledge from the datasets. Therefore, most methods do not define strategies to compose multiple continuous prompts. We propose two scalable solutions to introduce different variables in the design of continuous prompt labeler models (various demonstration examples or varying numbers of continuous prompt tokens). We expect that with these diverse continuous prompts, trained language models can fully learn different aspects of knowledge from the training dataset.

There are several automatic verbalizer methods that eliminate the need for human intervention and expertise to build mapping functions. We experiment with three types of automatic verbalizers: i) soft verbalizer Hambardzumyan et al. (2021), ii) prototypical verbalizer Cui et al. (2022), and iii) search-based verbalizer Schick et al. (2020b).

Cui et al. (2022) experimentally show the superiority of the prototypical verbalizer in a supervised learning environment. However, they did not conduct such experiments for SSL settings. Our experiment with the SSL PET method (details in Section 3.5) with different automatic verbalizers showed that the prototypical verbalizer performed better than the soft verbalizer and the search-based verbalizer on multiple datasets. Thus, we choose to use the prototypical verbalizer as a replacement for the manual verbalizer.

With the optimized embedding of the MASK token from PLM $m$ and the ground-truth labels $y$, the prototypical verbalizer learns the prototype vectors for each class using contrastive learning Oord et al. (2018). The prototypical verbalizer first initializes a prototype embedding for each class label and then uses the embedding of the MASK token as the instance embedding. It uses instance-instance loss $\mathcal{L}_{ins}$ to maximize intra-class similarity and minimize inter-class similarity. Similarly, it uses instance-prototype loss $\mathcal{L}_{proto}$ to maximize the similarity between the prototype and instances belonging to the same class and minimize the similarity of instances belonging to other classes. The probability distribution of the MASK token for each class is calculated by the cosine similarity between the instance embedding and each optimized prototype embedding. For inference, it assigns the class of the prototype vector to the instance with the highest probability score, which is computed by taking the similarity scores of the instance vector with the prototype vectors and normalizing them.

All model parameters to be optimized are randomly initialized. As mentioned in Section 2.2.2 and 2.3, we update the parameters in the continuous prompts and PLMs with the loss $\mathcal{L}_{c}$ and optimize the parameters in the verbalizers with the loss $\mathcal{L}_{ins}$ and $\mathcal{L}_{proto}$. Instead of summing all losses together, our training strategy is to first freeze the parameters in the prototypical verbalizer and then train the parameters in the reparameterization block and the PLM together with the cross-entropy loss $\mathcal{L}_{c}$. Then we freeze the learned parameters and train the parameters in the prototypical verbalizers with instance-instance loss $\mathcal{L}_{ins}$ and instance-prototype loss $\mathcal{L}_{proto}$. After training all labeler models and obtaining the class probability on the unlabeled dataset, we use $\mathcal{L}_{div}$ to fine-tune the final language model classifier. During inference, we do not rely on any prompt-based labeler models and directly use the final fine-tuned language model $\mathbf{F}$ to predict on the test dataset.

We experiment with five different datasets¹¹1We downloaded these datasets using the script provided by OpenPrompt https://github.com/thunlp/OpenPrompt: AG’s News Zhang et al. (2015a), Yahoo Answers Zhang et al. (2015b), MNLI (MultiNLI, Multi-Genre Natural Language Inference, Williams et al. (2018)), RTE (Recognizing Textual Entailment, Dagan et al. (2006)) and CB (CommitmentBank, de Marneffe et al. (2019)). AG’s News and Yahoo answers are topic classification (TC) datasets, while MNLI, RTE, and CB are natural language inference (NLI) datasets. In Table 1, we provide the number of distinct classes, the unlabeled dataset size used for SSL, and the test size for all five datasets. Details about the design of prompts and verbalizers can be found in Appendix A.

We perform multiple experiments in few-shot settings for all datasets. For few-shot experiments, we use $1,5,10,20$ examples per class for all datasets except for CB and RTE, where we experiment with $32$ examples to align with earlier research work Schick and Schütze (2021). We report the average accuracy for the evaluation across three runs of each experiment with three different random seeds.

We design several strong baseline experiments in addition to our proposed models and also perform an ablation study to show the superiority of our proposed models in multiple NLU tasks.

We use the RoBERTa-Large model  Liu et al. (2019) as our PLM for all of our experiments. We use AdamW as our optimizer with a learning rate of $1\mathrm{e}{-5}$ and a weight decay of $0.01$ with linear scheduler, batch size of $2$, and trained for $5$ epochs. The reparameterization block contains 2-layer bidirectional LSTM and 2 linear layers with ReLU activation function. The hidden dimension of the linear layer and LSTM layer is 768, as well as the hidden dimension of Roberta-Large. We train the parameters in the reparameterization block and the PLM together. For the prototypical verbalizer, we base our implementation on the Pytorch²²2https://pytorch.org/, Huggingface transformer³³3https://huggingface.co/, and OpenPrompt⁴⁴4https://github.com/thunlp/OpenPrompt frameworks Ding et al. (2021). For our Demo+Soft Tokens PET, each labeler model will learn 5 soft tokens with different demonstrations. For our Vary Soft Tokens PET, we prepare 5 prompts for each dataset and the number of soft tokens in each prompt ranges from 1 to 5.

To understand which automatic verbalizer is a better replacement for manual verbalizer, we first experiment with three automatic verbalizers: soft verbalizer Hambardzumyan et al. (2021); Liu et al. (2021c, b), search verbalizer Gao et al. (2021); Shin et al. (2020a); Schick et al. (2020a), and prototypical verbalizer Cui et al. (2022). For all of these experiments, we apply experimental setups similar to PET paper, but only replace the manual verbalizer with the automatic verbalizer Schick and Schütze (2021). Table 2 shows the average accuracy over three runs with three different seeds on different datasets with these verbalizers. From Table 2, the prototypical verbalizer shows better performance than other verbalizers for three (Yahoo, RTE, and MNLI) out of five datasets. The search verbalizer and soft verbalizer models perform better than the prototypical verbalizer model only on one dataset each. Since the prototypical verbalizer performs better than other verbalizers in majority of the datasets, we decided to use this as our automatic verbalizer.

With the prototypical verbalizer as our automatic verbalizer, we then experiment with our proposed methods for automatic prompt design. Table 3 shows our results on different datasets and tasks in the few-shot setting. Table 3 shows that by only replacing the manual verbalizer with the prototypical verbalizer (column Protoverb) and keeping other aspects of the experiment the same as the PET method, we can achieve slightly lower performance ($70.1$ average accuracy) compared to Manual PET ($71.4$ average accuracy) Schick and Schütze (2021). This shows that to eliminate human involvement in designing verbalizers, we can simply replace the manual verbalizer with the prototypical verbalizer with only a little performance sacrifice.

For our next set of experiments, we replace manual prompts with our proposed method, automatically creating multiple prompts. The first method (Demo+Soft Tokens PET), which adds randomly sampled demonstration examples from training data with a fixed number of trainable continuous prompt tokens with input, achieves better performance than Manual PET method. The next method (Vary Soft PET), in which we vary the number of continuous trainable tokens, also achieves better performance than Manual PET method. For topic classification tasks, under multiple few-shot settings, the average accuracy of Demo+Soft and Vary Soft PET are $77.0$ and $77.3$, respectively, while the average accuracy of Manual PET method is $77.1$. Similarly, for NLI datasets under different few-shot settings, the average accuracy of our Vary Soft PET method is $69.6$ and Demo+Soft Tokens PET method is $70.7$. Both of these results are better than Manual PET method ($67.7$). Furthermore, across all these datasets, Demo+Soft Tokens PET and Vary Soft PET achieve an average performance of $73.2$ and $72.6$, respectively. These results are better than Manual PET ($71.4$) method. This experiment shows that it is possible to completely eliminate human involvement and expertise in designing prompts and verbalizers for the SSL pipeline with even better performance.

We also observe that for the case of one-shot experiments with MNLI dataset, Demo + Soft PET method obtains an accuracy of $36.1$, which is much worse than other prompt baseline models. This may be due to randomly sampled $[demo]$ examples, as previous studies have shown that the choice of examples in the few-shot setting can result in high-variance performance Lu et al. (2021). In future work, we can utilize sentence embeddings to make intelligent decisions while selecting demonstration examples.

Cui et al. (2021) authors fine-tuned the pre-trained generative language model, BART, with a predefined template (${candidate\_span}$ is a ${entity\_type}$ entity) for NER classification. Wang et al. (2021) proposed Entailment as Few-shot Learner (EFT) method, which transforms classification tasks into natural language textual entailment tasks and then fine-tunes the LM. The transformation also makes it easy to leverage unsupervised contrastive data augmentation methods to add pairwise examples to the limited annotated data. This setting further showed an average 2.7% improvement in 15 different NLP tasks. In addition to using the prompts for supervised learning, PET is the SoTA method to adapt the manual prompts along with semi-supervised learning to obtain strong performance across multiple NLU tasks. Schick and Schütze (2021).

Shin et al. (2020a) used a gradient-guided search to find the discrete tokens for prompts based on task accuracy, initialize tokens, and then fine-tune the LM. For automatic label token selection, they first train a logistic regression classifier from the contextualized embedding of the MASK token and then predict the score from MLM’s output word embeddings. They select the top-k highest scoring words for each label. They showed better performance over manual prompting methods for sentiment classification and textual entailment tasks. Similarly, instead of using a gradient-guided search for prompt tokens, Li and Liang (2021) and Lester et al. (2021) attached prefix vectors and learned the embeddings for prefix vectors by keeping the LM model parameters frozen. Liu et al. (2021c) proposed P-tuning, which replaces the input embeddings of pre-trained language models with its differentiable output embeddings, using the pattern based on human design. Liu et al. (2021b) optimized and adapted the Prefix Tuning model for NLU. Vu et al. (2021) proposed to learn soft prompt embeddings from one or more source tasks and then transfer them to initialize the prompts for the target task. In addition, they also proposed an efficient retrieval approach to find task embeddings and predict the most transfarable source tasks for a given novel target task.

Several automatic verbalizers, such as search-based verbalizers, soft verbalizers, and prototypical verbalizers, have been proposed to automate the design of the verbalizer mapping function. Search-based verbalizers aim to find the appropriate tokens to replace human selection Schick et al. (2020a); Shin et al. (2020b); Gao et al. (2020). Both soft verbalizers and prototypical verbalizers learn trainable class or prototyope embeddings during the training process Cui et al. (2022); Zhang et al. (2021); Hambardzumyan et al. (2021).

Mahabadi et al. (2022) proposed a prompt-free method (PERFECT) to train the language model, which does not rely on manual commands and verbalizers. PERFECT reported performance similar to that of PET Schick and Schütze (2021) in a few-shot setting. However, they used a supervised learning setup and compared their results with the single labeler model with one prompt rather than the results from the final classifier. Here, we use a similar SSL setting to Schick and Schütze (2021) and report the results of the final classifier.