MixPoet: Diverse Poetry Generation via Learning
Controllable Mixed Latent Space

We first present a basic generator, one of our baselines, which is also a part of MixPoet. We adopt an effective structure similar to that in (?; ?).

Define $s_{i,j}$ as the corresponding GRU (?) decoder hidden state. Then the probability distribution of each $x_{i,j}$ to be generated is computed as:

where $[;]$ means concatenation; $e(\cdot)$ represents the embedding; $x_{<i}$ is the abbreviation of $x_{1},\dots,x_{i-1}$ (similar to $x_{i,<j}$); $o_{i}$ is a special length embedding (?) to control the length of each line; $f$ is a non-linear layer.

$g_{i-1}$ is a context vector to record so-far generated content in a poem and provide global information for the generator, which is used to keep context coherence and computed as:

where $\bm{0}$ is a zero vector and $d$ is the window size.

We introduce the semi-supervised framework of our model, which is adopted in our previous work (?). We first give the formalization based on a single factor $y$ for brevity and will incorporate more factors later.

Aiming at learning the conditional joint distribution $p(x,y|w)$, we can involve a latent variable $z$ and have $p(x,y|w)\!=\!\int p(x,y,z|w)dz$. Since style is coupled with semantics as mentioned in Sec. 1, rather than suppose the independence of $z$ and $y$, we decompose $p(x,y,z|w)$ as $p(x,y,z|w)=p(y|w)p(z|w,y)p(x|z,w,y)$. Such decomposition indicates how a poem is generated: if the user doesn’t provide any label, the model predicts an appropriate factor class by the keyword, then draws a sample of $z$ according to the required topic ($w$) and factor ($y$), and finally generates a poem ($x$). During this process we could manipulate both the topic and style of generated poems by separately specifying the keyword and factor class.

Then for labelled data, we can derive the lower bound:

where we approximate the true prior distribution $p(z|w,y)$ and posterior distribution $q(z|x,w,y)$ with a prior network $p_{\theta}(z|w,y)$ and a recognition network $q_{\phi}(z|x,w,y)$ respectively. $\theta$ and $\phi$ are corresponding parameter sets.

By optimizing Eq.(6), we reconstruct the poem $x$, and minimize the KL divergence of the posterior and prior distributions. Besides, we also incorporate a classifier $p_{\omega}(y|w)$ to predict appropriate factor classes when the user doesn’t provide any label. $\omega$ represents the parameters of classifiers.

Since the labelled data is too limited to train the model well, as (?), to utilize unlabelled poems, we treat the unobserved $y$ as another latent variable. In a similar vein, we can derive and maximize:

where another classifier $q_{\omega}(y|x,w)$ is trained to infer classes for unlabelled poems with Gumbel-softmax (?) during the training process.

Ultimately, the total semi-supervised loss is:

where we also add the classification loss to the first term to train the classifier $q_{\omega}(y|x,w)$ utilizing both supervised and unsupervised signals. $\alpha$ and $\beta$ are hyper-parameters.

Figure 2 diagrams our model. In detail, we take the whole poem $x$ as a long sequence and feed it into a bidirectional GRU. Then we concatenate the last forward and backward hidden states to form $h$, the vector representation of $x$. The classifiers are implemented with Multi-Layer Perceptron (MLP): $p_{\omega}(y|w)=softmax(MLP(e(w)))$ and $q_{\omega}(y|x,w)\!=\!softmax(MLP(e(w),h))$. We refer to $p_{\psi}(x|z,w,y)$ as the decoder (parameterized by $\psi$), which is just the basic generator introduced in Sec. 3.1, except that we set the initial decoder state as $s_{i,0}=f(e(w),o_{i},z,e(y))$ to involve the latent variable and the factor.

The formulas above only focus on a single factor. To incorporate $m$ factors, we can assume that the latent space can be disentangled into $m$ subspaces $z\!=\![z_{1};\cdots;z_{m}]$. Without loss of generality, we give the formulation when $m$=$2$. By further assuming the independence of influence factors and the conditional independence of these subspaces, we have $p(z|w,y)\!=\!p(z_{1}|w,y_{1})p(z_{2}|w,y_{2})$. Accordingly, we need to replace the classifiers in Sec. 3.2 with $p_{\omega}(y_{1}|w)$, $p_{\omega}(y_{2}|w)$, $q_{\omega}(y_{1}|x,w)$ and $q_{\omega}(y_{2}|x,w)$ to predict $y_{1}$ and $y_{2}$ respectively. This disentanglement indicates that we can independently draw $z_{1}$ and $z_{2}$ from corresponding factor-conditioned subspaces to form the whole latent variable. That is, we get a latent space which mixes the properties of different factors. We design two methods to learn such mixed latent space.

For MixPoet-IG, to alleviate the vanishing latent variable problem in VAE training, besides the annealing trick (?), we also add a BOW loss (?) to Eq.(8) to force $z$ to capture more global information. For MixPoet-AUS, since the discriminator is a crucial part for adversarial training, we adopt a powerful projection discriminator recently proposed in (?) and apply the spectral normalization (?) to the discriminator to stabilize the training process.

We mainly experiment on two typical factors: living experience and historical background. We discretize the first one into three classes: military career, countryside life and others; and the second one into two classes: prosperous times and troubled times. By mixing these factors, we can create six new styles. Then we build a labelled corpus called Chinese Quatrain Corpus with Factors (CQCF), which contains 49,451 poems, and each poem is labelled on at least one of the two factors. Statistics of CQCF are reported in Table 1. Besides, we also collect a Chinese Quatrain Corpus (CQC) as unlabelled data which comprises 117,392 poems. For CQC, we randomly select 4,500 poems for validation and testing, respectively, and the rest for training. For CQCF, we use 5% for validation, 5% for testing.

We use TextRank (?) to extract keywords from poems to build $<$keyword, poem$>$ pairs and $<$keyword, poem, labels$>$ triplets, as in (?).

We set the sizes of hidden state, context vector, latent variable, word embedding and factor embedding to 512, 512, 256, 256 and 64 respectively. The activation function is leaky ReLU for the discriminator and prior networks and is tanh for others. $d\!=\!3$ in Eq.(4); $\alpha\!=\!\beta=1$ in Eq.(8). Adam (?) with mini-batches (batch size=128) is used for optimization. To avoid overfitting, we also adopt dropout and $l_{2}$ norm regularization. For MixPoet-AUS, we update the discriminator five times per update of other parts. We first pre-train our model using both CQC and CQCF, and then fine-tune it with only CQCF. In testing, we adopt beam search (beam size=20) and apply explicit constraints to the search process to ensure that generated poems can meet the requirements of rhyme and rhythm. For fairness, all baselines share the same configuration.

We compare the following baselines¹¹1Since our model supports a single keyword, for fairness, we remove the keyword extension module of CVAE and fBasic.:

GT: ground truth, i.e. human-created poems. Basic: the generator introduced in Sec. 3.1. CVAE (?): a conditional VAE with a hybrid decoder for poetry generation. USPG (?): an unsupervised stylistic poetry generator which supports ten styles and can automatically infer an appropriate style by the input. MRL (?): a reinforcement learning model which achieves the so-far best inter-topic diversity. fBasic (?): a supervised stylistic poetry generator. We also pre-train fBaisc with both CQC and CQCF, and then fine-tune it with CQCF. fBasic takes a straightforward structure, but it represents the typical supervised paradigm of style control.

As in (?), we use Jaccard Similarity (JS) to evaluate diversity automatically. For inter-topic diversity, we generate 4,500 poems with different keywords but not any manually specified style/mixture, and then calculate JS of them. For intra-topic diversity, we calculate JS of poems generated with the same keyword but different specified styles. Besides, to prevent these models cheating by producing ill-formed content, we test the Language Model Score (LMS) (?) of generated poems. Higher LMS indicates moderate fluency closer to human-authored poetry.

Table 2 shows that on inter-topic diversity, our model outperforms most baselines and gets very close to MRL. Though with distinct keywords as input, most models tend to generate repetitive phrases (see Figure 5) which inevitably worsen diversity. MixPoet and USPG incorporate diverse styles to further differentiate generated poems. However, the unsupervised design of USPG results in indistinguishable and uninterpretable learned styles which have no explicit semantic meaning and are too similar. Consequently, even with fewer styles (3*2 vs. 10), MixPoet still surpasses USPG.

MRL obtains the best inter-topic diversity by penalizing high-frequency words but fails to achieve intra-topic diversity. If without extra post-processing, MRL (and Basic) can only generate the same poem by a given keyword (equivalent to intra-JS=100%). CVAE could produce somewhat different poems by utilizing different samples of $z$, but these poems heavily overlap with each other (intra-JS=38.2%). We can also see MixPoet-AUS gets better diversity than MixPoet-IG, as the former can learn more discriminable latent mixtures, we will analyse more in Sec. 4.7.

Compared to USPG and MRL, our model attributes diversity to the differences of various styles and interprets each style as a mixture of factor properties. Therefore, we also test if the generated poems are consistent with given factor classes.

For automatic evaluation, we generate 4,000 poems with each mixture and different keywords. Then we use a strong semi-supervised classifier (?), which achieves 0.87 and 0.74 F1 values for the two factors respectively, to measure the accuracy. For human evaluation, we generate 20 poems with each mixture (20*6 in total) and invite experts to identify the classes.

As shown in Figure 3, fBasic, the typical supervised method, performs the worst due to the quite limited and sparse labelled data. Benefiting from the semi-supervised structure, our model gets noticeable improvement. More than 80% and 60% of the generated poems meet specified classes of the two factors, respectively. Such results manifest that, to some extent, a poem generated by MixPoet can simultaneously express the properties of multiple factors.

Since automatic metrics (e.g., perplexity and BLEU) deviate from the human evaluation manner (?), we directly adopt human evaluation to assess quality. Following (?; ?; ?), we consider: Fluency (is the generated poem well-formed?), Context Coherence (is the poem as a whole thematically and logically structured?), Meaningfulness (does the poem convey certain messages?), Aesthetics (does the poem have some poetic and artistic beauties?), Topic Relevance (is the poem consistent with the given topic word?) Overall Quality (the general impression on the poem). Each of the six criteria is scored on a 5-point scale ranging from 1 to 5.

We use MixPoet-AUS, which achieves better results in the above assessments, for human quality evaluation and subsequent analyses and refer to it as MixPoet. Then for each model, we generate 40 poems with different randomly-selected keywords. For GT, we choose poems containing corresponding keywords. Therefore, we get 240 (40*6) poems in total. Then we invite ten experts to evaluate in a blind review manner. Each poem is randomly assigned to two experts, and we average the two scores to mitigate personal biases. We refer to (?; ?) for more details of the evaluation protocol.

As shown in Table 3 (Set 1), MixPoet gets notable improvement compared to other models. USPG is only better than Basic since it adopts a quite simple structure, even without any design for Coherence, which severely limits its performance. CVAE heavily relies on the support of multiple keywords. With a single keyword, it fails to produce meaningful contents, while our model can enrich semantic meanings by the mixed latent space. Despite obtaining the best inter-topic diversity, MRL may lose control of generated contents. Merely increasing TF-IDF could incur unexpected words digressing from topics and thus hurt quality.

Generally, we can find models achieving better diversity (MixPoet and MRL) outperform the others by a large margin since repetitive and generic words can damage poetic images and aesthetic features of generated poems, indicating that diversity also plays a crucial role in promoting quality.

It is noteworthy that generated poems take the risk of straying the given topic when constrained on one single style, because not all topics are compatible with every style. Therefore we also assess poems generated in Sec. 4.5. It can be seen from Table 3 (Set 2) that both fBasic and MixPoet performs somewhat worse on Relevance. Nonetheless, our model still gets acceptable results, since it utilizes the mixed latent space to capture more generalized properties of both factors and keywords, beyond simple labels.

In Figure 4 (a), we visualize points sampled from the prior distributions conditioned on the six mixtures. We can find MixPoet-AUS learns more discriminable latent representations, but Mixpoet-IG fails to distinguish different mixtures.

In Figure 4 (b), we vectorize poems by a neural language model and visualize them. We can see poems generated by our model, which mixes two factors (MC&PT), covers and bridges the two regions of human-authored poems, which indicates that our model successfully achieves the mixture not only on the latent space but also on generated poems.

From Figure 4 (c), we can observe that for Basic, CVAE and USPG, a few most frequent words account for a large proportion of generated contents, which leads to quite poor diversity. For instance, the most frequent five words cover over 10% of all contents generated by Basic. By contrast, our model alleviates this problem and gets a better balance in word distribution.

Figure 4 (d) demonstrates the effectiveness of the classifiers involved in our model. We can also find the styles of mixed factors are expressed through concrete contents (e.g., the use of images), which could support our claim that style is coupled with semantics.

As shown in Figure 5 (a), with two distinct keywords, Basic generates some repetitive words and even identical whole lines, causing poor diversity. By contrast, in Figure 5 (b), the poem generated by Mixpoet with MC&PT expresses great heroism and confidence in victory, while the other generated with MC&TT describes a scene of desolation and shows some loneliness. Besides, in ancient China, some weak dynasties were invaded by northern countries and thus moved their capitals to the south, with which many refugees also fled to the south. MixPoet may capture such events that are widely described by ancient poets and then generates “enemy’s warhorses march to the south” in the second poem (line 2). Though generated using the same keyword, these two poems present further diversity of thoughts and feelings.