Fine-grained Text Style Transfer with Diffusion-Based Language Models

Diffusion probabilistic models Ho et al. (2020) have became the state-of-the-art technique in visual generative tasks. By starting from random gaussian noise and gradual denoising, they are able to generate images that look realistic in details. Moreover, conditional diffusion models such as stable diffusion Rombach et al. (2022) are able to achieve detailed control over the generated output by conditioning on text, layouts, etc. The generated images are faithful to the text description or layouts, often to the finest details.

Analogically, researchers have tried to utilize the controllability of diffusion models to achieve more controllable language generation. For example, DiffuSeq Gong et al. (2022) applies diffusion models to sequence-sequence text generation tasks such as paraphrasing, question generation and text simplification; Diffusion-LM Li et al. (2022) combined diffusion models with language models to control language generation by specifying generation length, syntax tree, semantic context, etc. What made these diffusion-based language models impressive is that they are trained from scratch with zero external knowledge (i.e. no pre-trained word embeddings or model weights, no external grammar parsers, etc) and on very few data (on the order of $10^{5}$ tokens) compared to any large language models (for example, GPT-3’s Brown et al. (2020) training data is on the order $10^{11}$ tokens), so they have to learn representations at all levels (word embeddings, sentence structures, etc) from scratch with very limited data.

However, while the earlier tasks assessed on Diffusion-LM and DiffuSeq require a degree of control over the generated output, they are incapable of modifying the existing text to exhibit specific stylistic characteristics. In this paper, we would like to further examine the capabilities of diffusion-based language models on fine-grained text style transfer, an important task that requires more fine-grained control than the tasks from previous works on diffusion-based language modeling because it only allows changing the specified fine-grained stylistic properties of the input while leaving the rest unchanged. For example, "verb emphasis" is a fine-grained style transfer that requires the model to rewrite the sentence emphasizing a certain verb, without changing any other information that the original sentence conveys. In comparison, previous evaluation tasks such as controlling sequence length, semantic context, etc essentially control one aspect at a time and require no control over any other properties of generated text.

We use 13 non-lexical transfers from StylePTB Lyu et al. (2021) dataset, where there are at most a few thousand sentence pairs available for each transfer, as shown in Table 1. Since identifying the grammatical structure of the sentence can be very helpful for most of these transfers (such as active-to-passive), some previous methods (such as Neural QCFG Kim (2021)) utilizes external grammar parsers to gain such information. We trained a diffusion-based model on StylePTB data without any pre-trained weights or external grammar parsers. Therefore, our model has to start from zero grammar/linguistic knowledge and learn all of them from very limited training data (StylePTB only has 7719 sentences from Penn Tree Bank Marcus et al. (1993) plus their transferred outputs). Even under these hard conditions, our model still managed to outperform previous works that do utilize external weights or grammar parsers. Moreover, we also evaluate the capabilities of diffusion-based language models on performing multiple transfers using one single model and composing multiple learned transfers on a single sentence. We list our contributions as follows:

• •

  We trained a diffusion-based language model (adapted from DiffuSeq Gong et al. (2022)) that can perform fine-grained text style transfer from scratch with very limited training data and no external weights or tools. The model also supports multitasking and composing multiple fine-grained transfers.

• •

  Our model achieves state-of-the-art performance on fine-grained text style transfers in StylePTB. Our multitask model (i.e. one single model that can perform all 13 transfers) achieves best performance compared to previous works on the same tasks on 88 out of 91 metrics (7 metrics per transfer), and gets very close to human performance on tasks with easy and medium difficulties. We also evaluated our model on composition of multiple fine-grained transfers, and we achieved best performance on these tasks as well.

• •

  Thr/ough the evaluations, we demonstrated the extraordinary capabilities of diffusion-based language models in asserting extremely fine-grained control over generated text, and that this type of language model have great potential in controllable natural language generation under low-resource settings as it is able to achieve state-of-the-art performance with limited training data and no external knowledge.

We adapt DiffuSeq Gong et al. (2022) to be able to perform fine-grained text style transfer given a source sentence and specified transfer operation(s), as illustrated in Figure 1. We model the transfer as a conditional generation process, where the condition includes the source sentence and the specified transfer operation(s). We first define a set of special style tokens, one for each possible individual fine-grained transfer. If we wish to perform one or more transfer on the source sentence, we will prepend the corresponding special token(s) to the beginning of the source sentence to form the condition $S$.

We use BERT tokenizer to tokenize the input into discrete token ids, and adopt a token embedding layer to encode both the source (including prepended style tokens) and the ground truth target sentence (during training) to obtain the embedded source $Z^{S}$ and target $Z^{TRG}_{0}$. For the diffusion process, we use a transformer model to recover the target embedding. Both the diffusion transformer and the token embeddings are initialized randomly and jointly optimized. In other words, our model does not rely on any prior knowledge about our task or the English Language in general.

We use the simplified diffusion objective during training: for each input $(S,TRG)$ where $S$ is the source sentence (with style tokens) and $TRG$ is the ground truth target sentence, we randomly sample a step number $t$ from $1,2,...T$, where $T$ is the maximum number of steps, and add $t$ steps of random Gaussian noise to $Z^{TRG}_{0}$ following a linear diffusion schedule to obtain $Z^{TRG}_{t}$. We then concatenate $Z^{S}$ and $Z^{TRG}_{t}$ and input the concatenated sequence into our diffusion transformer, where we only take the output embeddings at the locations corresponding to $Z^{TRG}_{t}$ as $Z^{\prime TRG}_{0}$. Our training objective is simply going to be the MSE Loss between $Z^{TRG}_{0}$ and $Z^{\prime TRG}_{0}$.

During inference, we randomly initialize $Z^{\prime TRG}_{T}\sim N(0,1)$, and encode the condition (source sentence and style tokens) into $Z^{S}$. Then we concatenate them and use our transformer to predict a temporary $Z^{\prime TRG}_{0_{temp}}$, add $T-1$ steps of noise back to the temporary $Z^{\prime TRG}_{0_{temp}}$ to obtain $Z^{\prime TRG}_{T-1}$. We repeat this process until we get $Z^{\prime TRG}_{0}$. For each embedding in $Z^{\prime TRG}_{0}$, we find the closest embedding in our token embedding layer by cosine distance, and decode the embedding to that token. Then we combine the tokens to form the output sentence in natural language.

One significant limitation of our work is that we only explored the capabilities of diffusion-based language models under a challenging circumstance where it is not allowed to use pre-trained weights or grammar parsers, which means we did not utilize this kind of model to its full potential, so a future research direction could be exploring possible ways to further improve the model’s performance by leveraging pretrained weights or word embeddings, and train with enough data to find the full potential of these models.

Another limitation of our work is that we only explored one typical diffusion-based language model, so our conclusions may not generalize to special types of diffusion-based language models (such as ones that uses discrete state space). We also conducted all experiments using the exact same model architecture design. In the future, we plan to experiment with different architectures for the diffusion model, such as more sophisticated conditioning methods (currently we just concatenate the source to the target, but we would like to try other ways of conditioning on the source, such as cross attention, as these conditioning methods for diffusion models have promising performance in the image generation domain).

Lastly, we found that diffusion-based language models work well with limited data and no external knowledge or pre-trained weights, thus these models may have great potential under low-resource settings, but we didn’t apply them to any real low-resource settings (such as low-resource languages, rare domains, etc) in this paper, and we would like to do that in the future to explore the full potential of diffusion-based language models.

In this paper, we explored the capabilities of diffusion-based models on fine-grained text style transfer, a task that requires a high level of control over generated text, with no external knowledge or pre-trained weights and with very limited training data. Our diffusion-based language model, which builds upon DiffuSeq Gong et al. (2022), achieves state-of-the-art performance on all transfers as well as composition of transfers, outperforming all previous works on this dataset, including ones that uses pre-trained weights, word embeddings, and external grammar parsers. It is even on par with human performance on many transfers. Therefore, our model is a great step towards solving automated fine-grained text style transfer.

Moreover, our work, together with previous works such as Diffusion-LM Li et al. (2022), demonstrates that diffusion-based language models could have great potential in controllable text generation under low-resource settings. Under low-resource settings (such as rarely spoken language or uncommon tasks), it would be difficult to find existing large language models or pre-trained weights, and available training data will likely be very limited, so most approaches based on finetuning existing models or large amounts of training will not work well, and diffusion-based language models could be an alternative to consider.

This work is supported in part by grants from NSF IIS 1453651, NIH K12 NS080223, Cook Family Brain Tumor Research Fund, Mark Trauner Brain Research Fund: Zenkel Family Foundation, Ian’s Friends Foundation, and the Investigators Awards grant program of Precision Health at the University of Michigan. Any opinions, findings, conclusions, or recommendations expressed in this work are those of the author(s) and do not necessarily reflect the views of the NSF, NIH, Cook Family Brain Tumor Research Fund, Mark Trauner Brain Research Fund: Zenkel Family Foundation, Ian’s Friends Foundation, or Precision Health at the University of Michigan. We are grateful to the reviewers for their helpful review and feedback.